Pyodbc works in "base" but not other envs

Product Help Packages & Environments
1

I have a newly installed anaconda environment on redhat 8. I installed microsoft ODBC driver, kerberos,etc. all without problem. Then I installed Anaconda.

The pyodbc seemed automatically installed on “base”. I then created a new environment:
“conda create -n tstdev python=3.9”
then install pyodbc:
“conda activate tstdev; conda install pyodbc”

Strange thing happens. I can use pyodbc in “base” env, but not the other env.

(base) [admsagz@srvaz01c01-jobd ~]$ conda activate base
(base) [admsagz@srvaz01c01-jobd ~]$ python test-pyodbc.py
[(154, )] **=> pyodbc works in base **

(base) [admsagz@srvaz01c01-jobd ~]$ conda activate tstdev
(tstdev) [admsagz@srvaz01c01-jobd ~]$ python test-pyodbc.py
Traceback (most recent call last):
File “/home/[email protected]/test-pyodbc.py”, line 13, in
conn = pyodbc.connect(‘Driver={ODBC Driver 17 for SQL Server};Server=abc.com;Database=DAI_NLP_DEV;Trusted_Connection=yes;Mars_Connection=yes’)
pyodbc.Error: (‘01000’, “[01000] [unixODBC][Driver Manager]Can’t open lib ‘/opt/microsoft/msodbcsql17/lib64/libmsodbcsql-17.8.so.1.2’ : file not found (0) (SQLDriverConnect)”)

The “missing” file exists and accessible, otherwise the pyodbc on base would not have worked.

Would anyone provide some clue? Thanks in advance.

Regards,

Kevin

2

More information. The behavior reproduces if use “isql” came with anaconda:

(base) [admsagz@srvaz01c01-jobd ~]$ which isql
/apps/conda/anaconda3/bin/isql
(base) [admsagz@srvaz01c01-jobd ~]$ isql -v -k ‘Driver={ODBC Driver 17 for SQL Server};Server=abc.com;Database=DAI_NLP_DEV;Trusted_Connection=yes;Mars_Connection=yes’
±--------------------------------------+
| Connected! |
| |
| sql-statement |
| help [tablename] |
| quit |
| |
±--------------------------------------+
SQL> quit
(base) [admsagz@srvaz01c01-jobd ~]$ conda activate tstdev
(tstdev) [admsagz@srvaz01c01-jobd ~]$ which isql
/apps/conda/anaconda3/envs/tstdev/bin/isql
(tstdev) [admsagz@srvaz01c01-jobd ~]$ isql -v -k ‘Driver={ODBC Driver 17 for SQL Server};Server=abc.com;Database=DAI_NLP_DEV;Trusted_Connection=yes;Mars_Connection=yes’
[01000][unixODBC][Driver Manager]Can’t open lib ‘/opt/microsoft/msodbcsql17/lib64/libmsodbcsql-17.8.so.1.2’ : file not found
[ISQL]ERROR: Could not SQLDriverConnect
(tstdev) [admsagz@srvaz01c01-jobd ~]$

3

Here are ldd info for both isql:

(tstdev) [admsagz@srvaz01c01-jobd ~]$ ldd /apps/conda/anaconda3/envs/tstdev/bin/isql
linux-vdso.so.1 (0x00007ffc971f7000)
libodbc.so.2 => /apps/conda/anaconda3/envs/tstdev/bin/…/lib/libodbc.so.2 (0x00007fbe377df000)
libedit.so.0 => /apps/conda/anaconda3/envs/tstdev/bin/…/lib/libedit.so.0 (0x00007fbe377a3000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fbe3740e000)
libc.so.6 => /lib64/libc.so.6 (0x00007fbe37049000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007fbe36e45000)
libtinfow.so.6 => /apps/conda/anaconda3/envs/tstdev/bin/…/lib/./libtinfow.so.6 (0x00007fbe37757000)
/lib64/ld-linux-x86-64.so.2 (0x00007fbe3762e000)

(base) [admsagz@srvaz01c01-jobd ~]$ ldd /apps/conda/anaconda3/bin/isql
linux-vdso.so.1 (0x00007ffc38bea000)
libodbc.so.2 => /apps/conda/anaconda3/bin/…/lib/libodbc.so.2 (0x00007fbac711e000)
libedit.so.0 => /apps/conda/anaconda3/bin/…/lib/libedit.so.0 (0x00007fbac70e2000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fbac6d4d000)
libc.so.6 => /lib64/libc.so.6 (0x00007fbac6988000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007fbac6784000)
libtinfow.so.6 => /apps/conda/anaconda3/bin/…/lib/./libtinfow.so.6 (0x00007fbac7096000)
/lib64/ld-linux-x86-64.so.2 (0x00007fbac6f6d000)

4

(base) [admsagz@srvaz01c01-jobd ~]$ conda info

 active environment : base
active env location : /apps/conda/anaconda3
        shell level : 1
   user config file : /home/admsagz/.condarc

populated config files :
conda version : 4.10.3
conda-build version : 3.21.5
python version : 3.9.7.final.0
virtual packages : __linux=4.18.0=0
__glibc=2.28=0
__unix=0=0
__archspec=1=x86_64
base environment : /apps/conda/anaconda3 (read only)
conda av data dir : /apps/conda/anaconda3/etc/conda
conda av metadata url : None

      package cache : /apps/conda/anaconda3/pkgs
                      /home/admsagz/.conda/pkgs
   envs directories : /home/admsagz/.conda/envs
                      /apps/conda/anaconda3/envs
           platform : linux-64
         user-agent : conda/4.10.3 requests/2.26.0 CPython/3.9.7 Linux/4.18.0-305.25.1.el8_4.x86_64 rhel/8.4 glibc/2.28
            UID:GID : 756484642:756400513
         netrc file : None
       offline mode : False

(base) [admsagz@srvaz01c01-jobd ~]$ conda env list

conda environments:

base * /apps/conda/anaconda3
tstdev /apps/conda/anaconda3/envs/tstdev

5

Hi Kevin,

I would assume conda activate de-configured an environment variable somewhere. If you use the env command from within and without the activated environment, would you be able to determine which one it is?

Maybe these commands could help?

  • $ conda activate base
  • $ env > before
  • $ conda activate tstdev
  • $ env > after
  • $ diff before after
6

Hi Matt,

Thanks a lot for the reply. It seemed the differences were minimal:

(tstdev) [admsagz@srvaz01c01-jobd ~]$ diff before after
1c1
< CONDA_SHLVL=4
---
> CONDA_SHLVL=3
8c8
< CONDA_PREFIX=/apps/conda/anaconda3
---
> CONDA_PREFIX=/apps/conda/anaconda3/envs/tstdev
14d13
< CONDA_PREFIX_3=/apps/conda/anaconda3/envs/tstdev
21c20
< CONDA_PROMPT_MODIFIER=(base)
---
> CONDA_PROMPT_MODIFIER=(tstdev)
30,31c29,30
< PATH=/apps/conda/anaconda3/bin:/apps/conda/anaconda3/condabin:/home/[email protected]/.local/bin:/home/admsagz/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/mssql-tools/bin:/apps/conda/anaconda3/bin
< CONDA_DEFAULT_ENV=base
---
> PATH=/apps/conda/anaconda3/envs/tstdev/bin:/apps/conda/anaconda3/condabin:/home/admsagz/.local/bin:/home/admsagz/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/mssql-tools/bin:/apps/conda/anaconda3/bin
> CONDA_DEFAULT_ENV=tstdev

Another finding, the pyodbc worked when I created a new environment which cloned the “base”. It sounded like there were some pyodbc dependencies issue given the “base” had lots more packages installed than python3.9 + pyodbc. However, cloning is not a sufficient option for us because it would limit the python version to the base version.

conda create --name mybase --clone base
7

Doesn’t look like it’s environment variables to me then.

That clone thing is weird though. Sounds like maybe there are some packages installed into (base) that aren’t getting captured as dependencies.

Next step I’d try is to compare the output of conda list within the two environments to see if there’s anything different.

I’ve asked some others to see if they have any ideas. I’ll report back.

1 Like
8

I’d like to report back the findings. The missing dependency was krb5 in our case. One would expect python to use operating system’s krb5 if it’s not included in conda but it was not the case.

Regards

2 Likes