Error Installing Pyarrow with Python 3.7.4

Question

I'm developing a python script to deploy an Azure Function App. For this reason I can't use another Python version to make this easier.

In azure portal I get this error: Azure Function app pyarrow module not found

When I try to install it via VS Code with pip I get this error: Error installing Pyarrow

I managed to make this work using anaconda environment but since my goal is to make it run in an azure function I don't know how to solve this situation.

This is my requirements.txt:

arrow-cpp==0.17.*
pyarrow=0.17.*
adal==1.2.2
astroid==2.3.3
attrs==19.3.0
Automat==20.2.0
azure-applicationinsights==0.1.0
azure-batch==4.1.3
azure-common==1.1.24
azure-core==1.5.0
azure-cosmosdb-nspkg==2.0.2
azure-cosmosdb-table==1.0.6
azure-eventgrid==1.3.0
azure-functions==1.2.0
azure-graphrbac==0.40.0
azure-keyvault==1.1.0
azure-loganalytics==0.1.0
azure-mgmt==4.0.0
azure-mgmt-advisor==1.0.1
azure-mgmt-applicationinsights==0.1.1
azure-mgmt-authorization==0.50.0
azure-mgmt-batch==5.0.1
azure-mgmt-batchai==2.0.0
azure-mgmt-billing==0.2.0
azure-mgmt-cdn==3.1.0
azure-mgmt-cognitiveservices==3.0.0
azure-mgmt-commerce==1.0.1
azure-mgmt-compute==4.6.2
azure-mgmt-consumption==2.0.0
azure-mgmt-containerinstance==1.5.0
azure-mgmt-containerregistry==2.8.0
azure-mgmt-containerservice==4.4.0
azure-mgmt-cosmosdb==0.4.1
azure-mgmt-datafactory==0.6.0
azure-mgmt-datalake-analytics==0.6.0
azure-mgmt-datalake-nspkg==3.0.1
azure-mgmt-datalake-store==0.5.0
azure-mgmt-datamigration==1.0.0
azure-mgmt-devspaces==0.1.0
azure-mgmt-devtestlabs==2.2.0
azure-mgmt-dns==2.1.0
azure-mgmt-eventgrid==1.0.0
azure-mgmt-eventhub==2.6.0
azure-mgmt-hanaonazure==0.1.1
azure-mgmt-iotcentral==0.1.0
azure-mgmt-iothub==0.5.0
azure-mgmt-iothubprovisioningservices==0.2.0
azure-mgmt-keyvault==1.1.0
azure-mgmt-loganalytics==0.2.0
azure-mgmt-logic==3.0.0
azure-mgmt-machinelearningcompute==0.4.1
azure-mgmt-managementgroups==0.1.0
azure-mgmt-managementpartner==0.1.1
azure-mgmt-maps==0.1.0
azure-mgmt-marketplaceordering==0.1.0
azure-mgmt-media==1.0.0
azure-mgmt-monitor==0.5.2
azure-mgmt-msi==0.2.0
azure-mgmt-network==2.7.0
azure-mgmt-notificationhubs==2.1.0
azure-mgmt-nspkg==3.0.2
azure-mgmt-policyinsights==0.1.0
azure-mgmt-powerbiembedded==2.0.0
azure-mgmt-rdbms==1.9.0
azure-mgmt-recoveryservices==0.3.0
azure-mgmt-recoveryservicesbackup==0.3.0
azure-mgmt-redis==5.0.0
azure-mgmt-relay==0.1.0
azure-mgmt-reservations==0.2.1
azure-mgmt-resource==2.2.0
azure-mgmt-scheduler==2.0.0
azure-mgmt-search==2.1.0
azure-mgmt-servicebus==0.5.3
azure-mgmt-servicefabric==0.2.0
azure-mgmt-signalr==0.1.1
azure-mgmt-sql==0.9.1
azure-mgmt-storage==2.0.0
azure-mgmt-subscription==0.2.0
azure-mgmt-trafficmanager==0.50.0
azure-mgmt-web==0.35.0
azure-nspkg==3.0.2
azure-servicebus==0.21.1
azure-servicefabric==6.3.0.0
azure-servicemanagement-legacy==0.20.6
azure-storage==0.36.0
azure-storage-blob==12.3.1
azure-storage-common==1.4.2
azure-storage-file==1.4.0
azure-storage-file-datalake==12.0.1
azure-storage-queue==1.4.0
beautifulsoup4==4.8.1
bs4==0.0.1
certifi==2019.11.28
cffi==1.14.0
chardet==3.0.4
colorama==0.4.3
constantly==15.1.0
cryptography==2.8
cssselect==1.1.0
hyperlink==19.0.0
idna==2.7
incremental==17.5.0
isodate==0.6.0
isort==4.3.21
lazy-object-proxy==1.4.3
llvmlite==0.32.0
lxml==4.5.0
mccabe==0.6.1
msrest==0.6.11
msrestazure==0.6.2
numba==0.49.0
numpy==1.18.1
oauthlib==3.1.0
pandas==1.0.1
parsel==1.5.2
Protego==0.1.16
proxyscrape==0.3.0
pycparser==2.19
PyHamcrest==2.0.1
PyJWT==1.7.1
pylint==2.4.4
pyOpenSSL==19.1.0
python-dateutil==2.8.1
pytz==2019.3
requests==2.23.0
requests-oauthlib==1.3.0
six==1.14.0
soupsieve==1.9.5
thrift==0.13.0
urllib3==1.23
w3lib==1.21.0
wrapt==1.11.2
zope.interface==4.7.1

Answer 1

Thank you for your answer.

In fact, I think I can't change the environment because this venv is actually the environment used by Azure Functions. Is there any way I can use conda installed packages in the azure function

Answer 2

As you use conda as the package manager, you should also use it to install pyarrow and arrow-cpp using it. In your above output VSCode uses pip for the package management. You should consider reporting this as a bug to VSCode. Your current environment is detected as venv and not as conda environment as you can see in the Python environment box in the lower left.

The best workaround for now is to go to the terminal and manually type in conda install pyarrow=0.17 arrow-cpp=0.17. Note that you don't actually need to supply 0.17.* as conda automatically expands 0.17 to 0.17.*.