From: Thomas Perrot <thomas.perrot@bootlin.com>
To: Jose Quaresma <quaresma.jose@gmail.com>
Cc: bitbake-devel@lists.openembedded.org
Subject: Re: [bitbake-devel][PATCH] wget.py: always use the custom user agent
Date: Fri, 09 Feb 2024 17:04:28 +0100 [thread overview]
Message-ID: <3554214325be6023bc07eae6127f2cc445873f7b.camel@bootlin.com> (raw)
In-Reply-To: <CANPvuR=3rtSZfRaZbFUS4J+03o89fiURPzZc+abPSv52aFM-MA@mail.gmail.com>
[-- Attachment #1: Type: text/plain, Size: 3713 bytes --]
Hello Jose,
On Fri, 2024-02-09 at 15:52 +0000, Jose Quaresma wrote:
>
>
> Thomas Perrot via lists.openembedded.org
> <thomas.perrot=bootlin.com@lists.openembedded.org> escreveu (sexta,
> 9/02/2024 à(s) 15:25):
> > From: Thomas Perrot <thomas.perrot@bootlin.com>
> >
> > Add the "--user-agent" paramater in the wget base command to
> > perform all wget commands with this parameter, because a few
> > HTTP servers block requests with the default wget user agent.
> >
> > For example, "hg.openjdk.org" never send a response to requests
> > have been sent with wget:
> > wget https://hg.openjdk.org/jdk8u/jdk8u/archive/jdk8u272-ga.tar.bz2
> > https://hg.openjdk.org/jdk8u/jdk8u/archive/jdk8u272-ga.tar.bz2
> > Resolving hg.openjdk.org (hg.openjdk.org)... 23.54.129.73
> > Connecting to hg.openjdk.org (hg.openjdk.org)|23.54.129.73|:443...
> > connected.
> > HTTP request sent, awaiting response...
> >
> > Signed-off-by: Thomas Perrot <thomas.perrot@bootlin.com>
> > ---
> > lib/bb/fetch2/wget.py | 7 ++++---
> > 1 file changed, 4 insertions(+), 3 deletions(-)
> >
> > diff --git a/lib/bb/fetch2/wget.py b/lib/bb/fetch2/wget.py
> > index dc025800e659..e7c4cdd396b3 100644
> > --- a/lib/bb/fetch2/wget.py
> > +++ b/lib/bb/fetch2/wget.py
> > @@ -87,7 +87,8 @@ class Wget(FetchMethod):
> > if not ud.localfile:
> > ud.localfile = d.expand(urllib.parse.unquote(ud.host +
> > ud.path).replace("/", "."))
> >
> > - self.basecmd = d.getVar("FETCHCMD_wget") or "/usr/bin/env
> > wget -t 2 -T 30 --passive-ftp"
> > + self.basecmd = d.getVar("FETCHCMD_wget") \
> > + or "/usr/bin/env wget -t 2 -T 30 --passive-ftp --user-
> > agent='%s'" % (self.user_agent)
> >
> > if not self.check_certs(d):
> > self.basecmd += " --no-check-certificate"
> > @@ -454,7 +455,7 @@ class Wget(FetchMethod):
> > f = tempfile.NamedTemporaryFile()
> > with tempfile.TemporaryDirectory(prefix="wget-index-") as
> > workdir, tempfile.NamedTemporaryFile(dir=workdir, prefix="wget-
> > listing-") as f:
> > fetchcmd = self.basecmd
> > - fetchcmd += " -O " + f.name + " --user-agent='" +
> > self.user_agent + "' '" + uri + "'"
> > + fetchcmd += " -O '" + uri + "'"
> >
>
>
> Looks like the file name was removed unintentional.
Really sorry for this error, I'm sending a v2.
Kind regards,
Thomas Perrot
>
> Jose
>
> > try:
> > self._runwget(ud, d, fetchcmd, True,
> > workdir=workdir)
> > fetchresult = f.read()
> > @@ -492,7 +493,7 @@ class Wget(FetchMethod):
> > valid = 1
> > elif self._vercmp(version, newver) < 0:
> > version = newver
> > -
> > +
> > pupver = re.sub('_', '.', version[1])
> >
> > bb.debug(3, "*** %s -> UpstreamVersion = %s
> > (CurrentVersion = %s)" %
> >
> > -=-=-=-=-=-=-=-=-=-=-=-
> > Links: You receive all messages sent to this group.
> > View/Reply Online (#15853):
> > https://lists.openembedded.org/g/bitbake-devel/message/15853
> > Mute This Topic:
> > https://lists.openembedded.org/mt/104260370/5443093
> > Group Owner: bitbake-devel+owner@lists.openembedded.org
> > Unsubscribe:
> > https://lists.openembedded.org/g/bitbake-devel/unsub [thomas.perrot@bootlin.com
> > ]
> > -=-=-=-=-=-=-=-=-=-=-=-
> >
--
Thomas Perrot, Bootlin
Embedded Linux and kernel engineering
https://bootlin.com
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 659 bytes --]
prev parent reply other threads:[~2024-02-09 16:04 UTC|newest]
Thread overview: 3+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-02-09 15:25 [bitbake-devel][PATCH] wget.py: always use the custom user agent thomas.perrot
2024-02-09 15:52 ` Jose Quaresma
2024-02-09 16:04 ` Thomas Perrot [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=3554214325be6023bc07eae6127f2cc445873f7b.camel@bootlin.com \
--to=thomas.perrot@bootlin.com \
--cc=bitbake-devel@lists.openembedded.org \
--cc=quaresma.jose@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).