From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-ot1-x32b.google.com (mail-ot1-x32b.google.com [IPv6:2607:f8b0:4864:20::32b]) by sourceware.org (Postfix) with ESMTPS id 9CE593858D39 for ; Wed, 22 Sep 2021 09:50:03 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 9CE593858D39 Received: by mail-ot1-x32b.google.com with SMTP id l16-20020a9d6a90000000b0053b71f7dc83so2654344otq.7 for ; Wed, 22 Sep 2021 02:50:03 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=8JpSt8Vc7lGHeT9nNAg6FiBjRQpF8H7eFppYBs8B4nQ=; b=c5AWM/M6jsAQyciKwsd74qq0l1Yjo5TTQ5xHKTNyNej0cz/buFUsa5OujJtcQ8M9ML c71Ydns2R3GcltNJzfHtxLidhewaA6zxmZ9LzrlTLJ5ehyCDTIEXiTpKdfJ1i/M9EHlY DcJOTJJDXim3OePUhA/uK9LAGXh2otcpP9FBnLAIDbWf6FpK66vWR4OFtPKmwVr03jb/ Bgys4Wc1L3oD6X9Vq+fgwd5MgMhqHz0yxXEBqPvWo65z5TUuoG/AkyLD1rBfwaPg4KUl LSqP3rSzvYOwwh3vyNcrSR3YSRDayY4VsXw+oFcHgfFW3JO6hfIZjZ0JJsKvWrI1EHd5 Bd8w== X-Gm-Message-State: AOAM532bS9gSezP6I9J6ssZvGD+7eY/I70CDMlZLLjr9bJY6tlZ01bbd Vo546pVFpOWXp/NFywBg/KAMNNJ+tfkGfGJaXks= X-Google-Smtp-Source: ABdhPJxjpIjO+gSglVC7KWZLZhQubXZ3hhaeEnmuC/8CKwvMRCb9PL0R21nmlDJGHTp+GfDZ9ddWCf1WRxW9U6YBLPU= X-Received: by 2002:a05:6830:411d:: with SMTP id w29mr27373624ott.83.1632304202883; Wed, 22 Sep 2021 02:50:02 -0700 (PDT) MIME-Version: 1.0 References: <56f03ba0-a90d-b2e1-2c7d-20865f717334.ref@bellsouth.net> <56f03ba0-a90d-b2e1-2c7d-20865f717334@bellsouth.net> In-Reply-To: <56f03ba0-a90d-b2e1-2c7d-20865f717334@bellsouth.net> From: Russell VT Date: Wed, 22 Sep 2021 02:49:46 -0700 Message-ID: Subject: Re: Missing Python.h To: Dennis Putnam Cc: cygwin X-Spam-Status: No, score=-1.2 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, FREEMAIL_FROM, HTML_MESSAGE, KAM_LOTSOFHASH, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: cygwin@cygwin.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: General Cygwin discussions and problem reports List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Sep 2021 09:50:07 -0000 First off, this *probably* isn't a Cygwin problem ... but it looks like your environment is confused as it's using BOTH Python2 and Python3 modules to try to fulfill the requirements (including resources that have already been cached). For the most part, pip and pip3 can differentiate, but there's a "cart" and "horse" problem, as if you install things in a weird order at the system level, it may or may not do the right thing. But, I'd recommend "dumping" Python2, if you can at this point (it was EOL'd in December of 2020 and WILL NOT receive more updates, except for security ... and more-over, Python 3.7+ (approx) is going to demand newer SSL libraries that will REALLY confuse earlier versions). For working with Python (as a Python devops type), I generally recommend using 'pyenv' and 'pyenv-virtualenv' and trying to do *as little as possible* to modify the system-level Python ...this gets HARD with a system like Cygwin, where the generic user can generally overwrite system binaries, without any real sort of warning (and NO, UAC does NOT adequately fix anything). That also said, pyenv kinda really "fights" with Cygwin in some of the library placement (specifically things like FFI, IIRC, which is stored in a different library directory than it is, anywhere else I've found). Where I MIGHT start is to "Force Reinstall" the Python3 stuff from Setup. Look to see if requests_html is part of the Cygwin-supported modules, and use THAT... use the hell out of anything you see in the actual Python packages list, as those will at least be done RIGHT, and will leave you with more cycles to not worry about too much, except your development. For "Advanced" handling in Python, you're going to want to use "pyenv" or some other multi-python managers that are out there (virtualenv and virtualenv-wrapper are good, but ONLY manage the library path). Pretty much "pyenv" and "pipenv" are the top two, IIRC. I use pyenv, and haven't dug too deep in to pipenv, at this point. But, like I said, it's already tough enough to manage on older systems with older libraries (SSL, specifically, throws wrenches in to *everything*). Feel free to hit me up for other ideas... I write too much Python code, as it is, and on too many different environments (yes, some still do Python2.3 through 2.6, and it makes me want to shoot myself, sometimes... LOL). Hope that helps - Russell VT On Tue, Sep 21, 2021 at 11:38 AM Dennis Putnam wrote: > I am trying to install 'requests_html' and when it tries to do a compile > it fails because Python.h is missing. I have python2-devl installed. I > notice that it is looking for it in /pub which apparently does not > exist. Can someone help? TIA. > > Here is the entire 'pip' output: > > $ pip install requests_html > Collecting requests_html > Using cached requests_html-0.10.0-py3-none-any.whl (13 kB) > Collecting requests > Using cached requests-2.26.0-py2.py3-none-any.whl (62 kB) > Collecting w3lib > Using cached w3lib-1.22.0-py2.py3-none-any.whl (20 kB) > Collecting parse > Using cached parse-1.19.0.tar.gz (30 kB) > Collecting fake-useragent > Using cached fake-useragent-0.1.11.tar.gz (13 kB) > Collecting pyquery > Using cached pyquery-1.4.3-py3-none-any.whl (22 kB) > Collecting bs4 > Using cached bs4-0.0.1.tar.gz (1.1 kB) > Collecting pyppeteer>=0.0.14 > Using cached pyppeteer-0.2.6-py3-none-any.whl (83 kB) > Requirement already satisfied: tqdm<5.0.0,>=4.42.1 in > /usr/local/lib/python3.8/site-packages (from > pyppeteer>=0.0.14->requests_html) (4.62.3) > Requirement already satisfied: urllib3<2.0.0,>=1.25.8 in > /usr/local/lib/python3.8/site-packages (from > pyppeteer>=0.0.14->requests_html) (1.26.6) > Collecting appdirs<2.0.0,>=1.4.3 > Using cached appdirs-1.4.4-py2.py3-none-any.whl (9.6 kB) > Collecting importlib-metadata>=1.4 > Using cached importlib_metadata-4.8.1-py3-none-any.whl (17 kB) > Requirement already satisfied: pyee<9.0.0,>=8.1.0 in > /usr/local/lib/python3.8/site-packages (from > pyppeteer>=0.0.14->requests_html) (8.2.2) > Requirement already satisfied: websockets<10.0,>=9.1 in > /usr/local/lib/python3.8/site-packages (from > pyppeteer>=0.0.14->requests_html) (9.1) > Requirement already satisfied: zipp>=0.5 in > /usr/local/lib/python3.8/site-packages (from > importlib-metadata>=1.4->pyppeteer>=0.0.14->requests_html) ( 3.5.0) > Requirement already satisfied: beautifulsoup4 in > /usr/local/lib/python3.8/site-packages (from bs4->requests_html) (4.10.0) > Requirement already satisfied: soupsieve>1.2 in > /usr/local/lib/python3.8/site-packages (from > beautifulsoup4->bs4->requests_html) (2.2.1) > Collecting cssselect>0.7.9 > Using cached cssselect-1.1.0-py2.py3-none-any.whl (16 kB) > Collecting lxml>=2.1 > Using cached lxml-4.6.3.tar.gz (3.2 MB) > Collecting idna<4,>=2.5 > Using cached idna-3.2-py3-none-any.whl (59 kB) > Collecting certifi>=2017.4.17 > Using cached certifi-2021.5.30-py2.py3-none-any.whl (145 kB) > Collecting charset-normalizer~=2.0.0 > Using cached charset_normalizer-2.0.6-py3-none-any.whl (37 kB) > Requirement already satisfied: six>=1.4.1 in > /usr/local/lib/python3.8/site-packages (from w3lib->requests_html) (1.16.0) > Using legacy 'setup.py install' for bs4, since package 'wheel' is not > installed. > Using legacy 'setup.py install' for fake-useragent, since package > 'wheel' is not installed. > Using legacy 'setup.py install' for parse, since package 'wheel' is not > installed. > Using legacy 'setup.py install' for lxml, since package 'wheel' is not > installed. > Installing collected packages: lxml, importlib-metadata, idna, > cssselect, charset-normalizer, certifi, appdirs, w3lib, requests, > pyquery, pyppeteer, > parse, fake-useragent, bs4, requests-html > Running setup.py install for lxml ... error > ERROR: Command errored out with exit status 1: > command: /usr/bin/python -u -c 'import io, os, sys, setuptools, > tokenize; sys.argv[0] = > '"'"'/tmp/pip-install-mp64a75l/lxml_61e1128ed27f4063936a > 7a21e44c273e/setup.py'"'"'; > __file__='"'"'/tmp/pip-install-mp64a75l/lxml_61e1128ed27f4063936a7a21e44c273e/setup.py'"'"';f > > = getattr(tokenize, '"'"'open'"'"', open)(__file__) if > os.path.exists(__file__) else io.StringIO('"'"'from setuptools import > setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', > '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' > install --record /tmp/pip-record-qafwouur/install-record.txt > --single-version-externally-managed --compile --install-headers > /usr/local/include/python3.8/lxml > cwd: > /tmp/pip-install-mp64a75l/lxml_61e1128ed27f4063936a7a21e44c273e/ > Complete output (86 lines): > Building lxml version 4.6.3. > Building without Cython. > Building against libxml2 2.9.10 and libxslt 1.1.29 > running install > running build > running build_py > creating build > creating build/lib.cygwin-3.2.0-x86_64-3.8 > creating build/lib.cygwin-3.2.0-x86_64-3.8/lxml > copying src/lxml/builder.py -> build/lib.cygwin-3.2.0-x86_64-3.8/lxml > copying src/lxml/cssselect.py -> > build/lib.cygwin-3.2.0-x86_64-3.8/lxml > copying src/lxml/doctestcompare.py -> > build/lib.cygwin-3.2.0-x86_64-3.8/lxml > copying src/lxml/ElementInclude.py -> > build/lib.cygwin-3.2.0-x86_64-3.8/lxml > copying src/lxml/pyclasslookup.py -> > build/lib.cygwin-3.2.0-x86_64-3.8/lxml > copying src/lxml/sax.py -> build/lib.cygwin-3.2.0-x86_64-3.8/lxml > copying src/lxml/usedoctest.py -> > build/lib.cygwin-3.2.0-x86_64-3.8/lxml > copying src/lxml/_elementpath.py -> > build/lib.cygwin-3.2.0-x86_64-3.8/lxml > copying src/lxml/__init__.py -> build/lib.cygwin-3.2.0-x86_64-3.8/lxml > creating build/lib.cygwin-3.2.0-x86_64-3.8/lxml/includes > copying src/lxml/includes/__init__.py -> > build/lib.cygwin-3.2.0-x86_64-3.8/lxml/includes > creating build/lib.cygwin-3.2.0-x86_64-3.8/lxml/html > copying src/lxml/html/builder.py -> > build/lib.cygwin-3.2.0-x86_64-3.8/lxml/html > copying src/lxml/html/clean.py -> > build/lib.cygwin-3.2.0-x86_64-3.8/lxml/html > copying src/lxml/html/defs.py -> > build/lib.cygwin-3.2.0-x86_64-3.8/lxml/html > copying src/lxml/html/diff.py -> > build/lib.cygwin-3.2.0-x86_64-3.8/lxml/html > copying src/lxml/html/ElementSoup.py -> > build/lib.cygwin-3.2.0-x86_64-3.8/lxml/html > copying src/lxml/html/formfill.py -> > build/lib.cygwin-3.2.0-x86_64-3.8/lxml/html > copying src/lxml/html/html5parser.py -> > build/lib.cygwin-3.2.0-x86_64-3.8/lxml/html > copying src/lxml/html/soupparser.py -> > build/lib.cygwin-3.2.0-x86_64-3.8/lxml/html > copying src/lxml/html/usedoctest.py -> > build/lib.cygwin-3.2.0-x86_64-3.8/lxml/html > copying src/lxml/html/_diffcommand.py -> > build/lib.cygwin-3.2.0-x86_64-3.8/lxml/html > copying src/lxml/html/_html5builder.py -> > build/lib.cygwin-3.2.0-x86_64-3.8/lxml/html > copying src/lxml/html/_setmixin.py -> > build/lib.cygwin-3.2.0-x86_64-3.8/lxml/html > copying src/lxml/html/__init__.py -> > build/lib.cygwin-3.2.0-x86_64-3.8/lxml/html > creating build/lib.cygwin-3.2.0-x86_64-3.8/lxml/isoschematron > copying src/lxml/isoschematron/__init__.py -> > build/lib.cygwin-3.2.0-x86_64-3.8/lxml/isoschematron > copying src/lxml/etree.h -> build/lib.cygwin-3.2.0-x86_64-3.8/lxml > copying src/lxml/etree_api.h -> build/lib.cygwin-3.2.0-x86_64-3.8/lxml > copying src/lxml/lxml.etree.h -> > build/lib.cygwin-3.2.0-x86_64-3.8/lxml > copying src/lxml/lxml.etree_api.h -> > build/lib.cygwin-3.2.0-x86_64-3.8/lxml > copying src/lxml/includes/c14n.pxd -> > build/lib.cygwin-3.2.0-x86_64-3.8/lxml/includes > copying src/lxml/includes/config.pxd -> > build/lib.cygwin-3.2.0-x86_64-3.8/lxml/includes > copying src/lxml/includes/dtdvalid.pxd -> > build/lib.cygwin-3.2.0-x86_64-3.8/lxml/includes > copying src/lxml/includes/etreepublic.pxd -> > build/lib.cygwin-3.2.0-x86_64-3.8/lxml/includes > copying src/lxml/includes/htmlparser.pxd -> > build/lib.cygwin-3.2.0-x86_64-3.8/lxml/includes > copying src/lxml/includes/relaxng.pxd -> > build/lib.cygwin-3.2.0-x86_64-3.8/lxml/includes > copying src/lxml/includes/schematron.pxd -> > build/lib.cygwin-3.2.0-x86_64-3.8/lxml/includes > copying src/lxml/includes/tree.pxd -> > build/lib.cygwin-3.2.0-x86_64-3.8/lxml/includes > copying src/lxml/includes/uri.pxd -> > build/lib.cygwin-3.2.0-x86_64-3.8/lxml/includes > copying src/lxml/includes/xinclude.pxd -> > build/lib.cygwin-3.2.0-x86_64-3.8/lxml/includes > copying src/lxml/includes/xmlerror.pxd -> > build/lib.cygwin-3.2.0-x86_64-3.8/lxml/includes > copying src/lxml/includes/xmlparser.pxd -> > build/lib.cygwin-3.2.0-x86_64-3.8/lxml/includes > copying src/lxml/includes/xmlschema.pxd -> > build/lib.cygwin-3.2.0-x86_64-3.8/lxml/includes > copying src/lxml/includes/xpath.pxd -> > build/lib.cygwin-3.2.0-x86_64-3.8/lxml/includes > copying src/lxml/includes/xslt.pxd -> > build/lib.cygwin-3.2.0-x86_64-3.8/lxml/includes > copying src/lxml/includes/__init__.pxd -> > build/lib.cygwin-3.2.0-x86_64-3.8/lxml/includes > copying src/lxml/includes/etree_defs.h -> > build/lib.cygwin-3.2.0-x86_64-3.8/lxml/includes > copying src/lxml/includes/lxml-version.h -> > build/lib.cygwin-3.2.0-x86_64-3.8/lxml/includes > creating > build/lib.cygwin-3.2.0-x86_64-3.8/lxml/isoschematron/resources > creating > build/lib.cygwin-3.2.0-x86_64-3.8/lxml/isoschematron/resources/rng > copying src/lxml/isoschematron/resources/rng/iso-schematron.rng -> > build/lib.cygwin-3.2.0-x86_64-3.8/lxml/isoschematron/resources/rng > creating > build/lib.cygwin-3.2.0-x86_64-3.8/lxml/isoschematron/resources/xsl > copying src/lxml/isoschematron/resources/xsl/RNG2Schtrn.xsl -> > build/lib.cygwin-3.2.0-x86_64-3.8/lxml/isoschematron/resources/xsl > copying src/lxml/isoschematron/resources/xsl/XSD2Schtrn.xsl -> > build/lib.cygwin-3.2.0-x86_64-3.8/lxml/isoschematron/resources/xsl > creating > build/lib.cygwin-3.2.0-x86_64-3.8/lxml/isoschematron/resources/xsl/iso-schematron-xslt1 > > > copying > src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_abstract_expand.xsl > > -> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/isoschematron > /resources/xsl/iso-schematron-xslt1 > copying > src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_dsdl_include.xsl > > -> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/isoschematron/re > sources/xsl/iso-schematron-xslt1 > copying > src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_schematron_message.xsl > > -> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/isoschemat > ron/resources/xsl/iso-schematron-xslt1 > copying > src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_schematron_skeleton_for_xslt1.xsl > > -> build/lib.cygwin-3.2.0-x86_64-3.8/lxml > /isoschematron/resources/xsl/iso-schematron-xslt1 > copying > src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_svrl_for_xslt1.xsl > > -> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/isoschematron/ > resources/xsl/iso-schematron-xslt1 > copying > src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/readme.txt -> > build/lib.cygwin-3.2.0-x86_64-3.8/lxml/isoschematron/resources/xs > l/iso-schematron-xslt1 > running build_ext > building 'lxml.etree' extension > creating build/temp.cygwin-3.2.0-x86_64-3.8 > creating build/temp.cygwin-3.2.0-x86_64-3.8/src > creating build/temp.cygwin-3.2.0-x86_64-3.8/src/lxml > gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall > -ggdb -O2 -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 > -fstack-protector-strong --param=ssp-buffer-size=4 -DOPENSSL_NO_SSL3=1 > -fdebug-prefix-map=/pub/devel/python/python38/python38-3.8.10-1.x86_64/build=/usr/src/debug/python38-3.8.10-1 > > -fdebug-prefix-map=/pub/devel/python/python38/python38-3.8.10-1.x86_64/src/Python-3.8.10=/usr/src/debug/python38-3.8.10-1 > > -ggdb -O2 -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 > -fstack-protector-strong --param=ssp-buffer-size=4 -DOPENSSL_NO_SSL3=1 > -fdebug-prefix-map=/pub/devel/python/python38/python38-3.8.10-1.x86_64/build=/usr/src/debug/python38-3.8.10-1 > > -fdebug-prefix-map=/pub/devel/python/python38/python38-3.8.10-1.x86_64/src/Python-3.8.10=/usr/src/debug/python38-3.8.10-1 > > -DCYTHON_CLINE_IN_TRACEBACK=0 -I/usr/include/libxml2 -Isrc > -Isrc/lxml/includes -I/usr/include/python3.8 -c src/lxml/etree.c -o > build/temp.cygwin-3.2.0-x86_64-3.8/src/lxml/etree.o -w > src/lxml/etree.c:97:10: fatal error: Python.h: No such file or > directory > 97 | #include "Python.h" > | ^~~~~~~~~~ > compilation terminated. > Compile failed: command 'gcc' failed with exit status 1 > creating tmp > cc -I/usr/include/libxml2 -I/usr/include/libxml2 -c > /tmp/xmlXPathInit4efcky6t.c -o tmp/xmlXPathInit4efcky6t.o > cc tmp/xmlXPathInit4efcky6t.o -lxml2 -o a.out.exe > error: command 'gcc' failed with exit status 1 > ---------------------------------------- > ERROR: Command errored out with exit status 1: /usr/bin/python -u -c > 'import io, os, sys, setuptools, tokenize; sys.argv[0] = > '"'"'/tmp/pip-install-mp64a75l/lxml_61e1128ed27f4063936a7a21e44c273e/setup.py'"'"'; > > __file__='"'"'/tmp/pip-install-mp64a75l/lxml_61e1128ed27f4063936a7a21e44c273e/setup.py'"'"';f > > = getattr(tokenize, '"'"'open'"'"', open)(__file__) if > os.path.exists(__file__) else io.StringIO('"'"'from setuptools import > setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', > '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' > install --record /tmp/pip-record-qafwouur/install-record.txt > --single-version-externally-managed --compile --install-headers > /usr/local/include/python3.8/lxml Check the logs for full command output. > > > -- > Problem reports: https://cygwin.com/problems.html > FAQ: https://cygwin.com/faq/ > Documentation: https://cygwin.com/docs.html > Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple > -- Russell M. Van Tassell