Interface for modifying Windows environment variables from Python

Question

How can I persistently modify the Windows environment variables from a Python script? (it's the setup.py script)

I'm looking for a standard function or module to use for this. I'm already familiar with the registry way of doing it, but any comments regarding that are also welcome.

Answer 1

This Python-script[*] attempts to modify the GLOBAL env-vars in registry, if no-permissions falls-back to user's registry, and then notifies all windows about the change:

"""
Show/Modify/Append registry env-vars (ie `PATH`) and notify Windows-applications to pickup changes.

First attempts to show/modify HKEY_LOCAL_MACHINE (all users), and 
if not accessible due to admin-rights missing, fails-back 
to HKEY_CURRENT_USER.
Write and Delete operations do not proceed to user-tree if all-users succeed.

Syntax: 
    {prog}                  : Print all env-vars. 
    {prog}  VARNAME         : Print value for VARNAME. 
    {prog}  VARNAME   VALUE : Set VALUE for VARNAME. 
    {prog}  +VARNAME  VALUE : Append VALUE in VARNAME delimeted with ';' (i.e. used for `PATH`). 
    {prog}  -VARNAME        : Delete env-var value. 

Note that the current command-window will not be affected, 
changes would apply only for new command-windows.
"""

import winreg
import os, sys, win32gui, win32con

def reg_key(tree, path, varname):
    return '%s\%s:%s' % (tree, path, varname) 

def reg_entry(tree, path, varname, value):
    return '%s=%s' % (reg_key(tree, path, varname), value)

def query_value(key, varname):
    value, type_id = winreg.QueryValueEx(key, varname)
    return value

def show_all(tree, path, key):
    i = 0
    while True:
        try:
            n,v,t = winreg.EnumValue(key, i)
            print(reg_entry(tree, path, n, v))
            i += 1
        except OSError:
            break ## Expected, this is how iteration ends.

def notify_windows(action, tree, path, varname, value):
    win32gui.SendMessage(win32con.HWND_BROADCAST, win32con.WM_SETTINGCHANGE, 0, 'Environment')
    print("---%s %s" % (action, reg_entry(tree, path, varname, value)))

def manage_registry_env_vars(varname=None, value=None):
    reg_keys = [
        ('HKEY_LOCAL_MACHINE', r'SYSTEM\CurrentControlSet\Control\Session Manager\Environment'),
        ('HKEY_CURRENT_USER', r'Environment'),
    ]
    for (tree_name, path) in reg_keys:
        tree = eval('winreg.%s'%tree_name)
        try:
            with winreg.ConnectRegistry(None, tree) as reg:
                with winreg.OpenKey(reg, path, 0, winreg.KEY_ALL_ACCESS) as key:
                    if not varname:
                        show_all(tree_name, path, key)
                    else:
                        if not value:
                            if varname.startswith('-'):
                                varname = varname[1:]
                                value = query_value(key, varname)
                                winreg.DeleteValue(key, varname)
                                notify_windows("Deleted", tree_name, path, varname, value)
                                break  ## Don't propagate into user-tree.
                            else:
                                value = query_value(key, varname)
                                print(reg_entry(tree_name, path, varname, value))
                        else:
                            if varname.startswith('+'):
                                varname = varname[1:]
                                value = query_value(key, varname) + ';' + value
                            winreg.SetValueEx(key, varname, 0, winreg.REG_EXPAND_SZ, value)
                            notify_windows("Updated", tree_name, path, varname, value)
                            break  ## Don't propagate into user-tree.
        except PermissionError as ex:
            print("!!!Cannot access %s due to: %s" % 
                    (reg_key(tree_name, path, varname), ex))
        except FileNotFoundError as ex:
            print("!!!Cannot find %s due to: %s" % 
                    (reg_key(tree_name, path, varname), ex))

if __name__=='__main__':
    args = sys.argv
    argc = len(args)
    if argc > 3:
        print(__doc__.format(prog=args[0]))
        sys.exit()

    manage_registry_env_vars(*args[1:])

Below are some usage examples, assuming it has been saved in a file called setenv.py somewhere in your current path. Note that in these examples i didn't have admin-rights, so the changes affected only my local user's registry tree:

> REM ## Print all env-vars
> setenv.py
!!!Cannot access HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session   Manager\Environment:PATH due to: [WinError 5] Access is denied
HKEY_CURRENT_USER\Environment:PATH=...
...

> REM ## Query env-var:
> setenv.py PATH C:\foo
!!!Cannot access HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session   Manager\Environment:PATH due to: [WinError 5] Access is denied
!!!Cannot find HKEY_CURRENT_USER\Environment:PATH due to: [WinError 2] The system cannot find the file specified

> REM ## Set env-var:
> setenv.py PATH C:\foo
!!!Cannot access HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session   Manager\Environment:PATH due to: [WinError 5] Access is denied
---Set HKEY_CURRENT_USER\Environment:PATH=C:\foo

> REM ## Append env-var:
> setenv.py +PATH D:\Bar
!!!Cannot access HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session   Manager\Environment:PATH due to: [WinError 5] Access is denied
---Set HKEY_CURRENT_USER\Environment:PATH=C:\foo;D:\Bar

> REM ## Delete env-var:
> setenv.py -PATH
!!!Cannot access HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session   Manager\Environment:PATH due to: [WinError 5] Access is denied
---Deleted HKEY_CURRENT_USER\Environment:PATH

[*] Adapted from: http://code.activestate.com/recipes/416087-persistent-environment-variables-on-windows/

Answer 2

It must have been a millennium ago that I've tried to change the environment of the current DOS session by means of a program. The problem is: that program runs within its own DOS shell, so it has to operate on its parent environment. It takes a walk, starting at the DOS Info Block, all along the chain of Memory Control Blocks, to find the location of that parent environment. Once I had found out how to do this, my need for manipulating environment variables had vanished. I'll give you the Turbo Pascal code below, but I guess there are at least three better ways to do the trick:

1. Create a batch file that: (a) calls a Python script (or whatever) that generates a temporay batch file containing the appropriate SET commands; (b) calls the temporary batch file (the SET commands are executed in the current shell); and (c) removes the temporary batch file.

2. Create a Python script that writes something like "VAR1=val1\nVAR2=val2\nVAR3=val3\n" to STDOUT. Use it this way in your batch file:

   for /f "delims=|" %%X in ('callYourPythonScript') do set %%X

   et voilà: variables VAR1, VAR2 and VAR3 have been given a value.

3. Modify the Windows Registry and broadcast the setting change as described here by Alexander Prokofyev.

And here goes the Pascal code (you may need a Dutch dictionary and a Pascal programming book) of a program that just reports memory locations. It still seems to work under Windows XP, be it reporting we're running DOS 5.00. This is only a first beginning, there's a lot of low level programming to do in order to manipulate the selected environment. And as the pointer structure may seem correct, I'm not so sure if the environment model of 1994 still holds these days...

program MCBKETEN;
uses dos, HexConv;

{----------------------------------------------------------------------------}
{  Programma: MCBKETEN.EXE                                                   }
{  Broncode : MCBKETEN.PAS                                                   }
{  Doel     : Tocht langs de MCB's met rapportage                            }
{  Datum    : 11 januari 1994                                                }
{  Auteur   : Meindert Meindertsma                                           }
{  Versie   : 1.00                                                           }
{----------------------------------------------------------------------------}

type
   MCB_Ptr     = ^MCB;
{  MCB_PtrPtr  = ^MCB_Ptr;  vervallen wegens DOS 2.11 -- zie verderop }
   MCB         = record
                    Signatuur    : char;
                    Eigenaar     : word;
                    Paragrafen   : word;
                    Gereserveerd : array[1..3] of byte;
                    Naam         : array[1..8] of char;
                 end;
   BlokPtr     = ^BlokRec;
   BlokRec     = record
                    Vorige       : BlokPtr;
                    DitSegment,
                    Paragrafen   : word;
                    Signatuur    : string[6];
                    Eigenaar,
                    Omgeving     : word;
                    Functie      : String4;
                    Oorsprong,
                    Pijl         : char;
                    KorteNaam    : string[8];
                    LangeNaam    : string;
                    Volgende     : BlokPtr;
                 end;
   PSP_Ptr     = ^PSP;
   PSP         = record
                    Vulsel1      : array[1..44] of byte;
                    Omgeving     : word;
                    Vulsel2      : array[47..256] of byte;
                 end;

var
   Zone                  : string[5];
   ProgGevonden,
   EindeKeten,
   Dos3punt2             : boolean;
   Regs                  : registers;
   ActMCB                : MCB_Ptr;
   EersteSchakel, Schakel,
   LaatsteSchakel        : BlokPtr;
   ActPSP                : PSP_Ptr;
   EersteProg,
   Meester, Ouder,
   TerugkeerSegment,
   TerugkeerOffset,
   TerugkeerSegment2,
   OuderSegment          : word;
   Specificatie          : string[8];
   ReleaseNummer         : string[2];
   i                     : byte;


{----------------------------------------------------------------------------}
{  PROCEDURES EN FUNCTIES                                                    }
{----------------------------------------------------------------------------}

function Coda (Omgeving : word; Paragrafen : word) : string;

var
   i            : longint;
   Vorige, Deze : char;
   Streng       : string;

begin
   i    := 0;
   Deze := #0;
   repeat
      Vorige := Deze;
      Deze   := char (ptr (Omgeving, i)^);
      inc (i);
   until ((Vorige = #0) and (Deze = #0)) or (i div $10 >= Paragrafen);
   if (i + 3) div $10 < Paragrafen then begin
      Vorige := char (ptr (Omgeving, i)^);
      inc (i);
      Deze   := char (ptr (Omgeving, i)^);
      inc (i);
      if (Vorige = #01) and (Deze = #0) then begin
         Streng := '';
         Deze   := char (ptr (Omgeving, i)^);
         inc (i);
         while (Deze <> #0) and (i div $10 < Paragrafen) do begin
            Streng := Streng + Deze;
            Deze   := char (ptr (Omgeving, i)^);
            inc (i);
         end;
         Coda := Streng;
      end
      else Coda := '';
   end
   else Coda := '';
end {Coda};


{----------------------------------------------------------------------------}
{  HOOFDPROGRAMMA                                                            }
{----------------------------------------------------------------------------}

BEGIN
  {----- Initiatie -----}
   Zone            := 'Lower';
   ProgGevonden    := FALSE;
   EindeKeten      := FALSE;
   Dos3punt2       := (dosversion >= $1403) and (dosversion <= $1D03);
   Meester         := $0000;
   Ouder           := $0000;
   Specificatie[0] := #8;
   str (hi (dosversion) : 2, ReleaseNummer);
   if ReleaseNummer[1] = ' ' then ReleaseNummer[1] := '0';

  {----- Pointer naar eerste MCB ophalen ------}
   Regs.AH := $52;  { functie $52 geeft adres van DOS Info Block in ES:BX }
   msdos (Regs);
{  ActMCB := MCB_PtrPtr (ptr (Regs.ES, Regs.BX - 4))^;  NIET onder DOS 2.11  }
   ActMCB := ptr (word (ptr (Regs.ES, Regs.BX - 2)^), $0000);

  {----- MCB-keten doorlopen -----}
   new (EersteSchakel);
   EersteSchakel^.Vorige := nil;
   Schakel               := EersteSchakel;
   repeat
      with Schakel^ do begin
         DitSegment := seg (ActMCB^);
         Paragrafen := ActMCB^.Paragrafen;
         if DitSegment + Paragrafen >= $A000 then
            Zone    := 'Upper';
         Signatuur  := Zone + ActMCB^.Signatuur;
         Eigenaar   := ActMCB^.Eigenaar;
         ActPSP     := ptr (Eigenaar, 0);
         if not ProgGevonden then EersteProg := DitSegment + 1;
         if Eigenaar >= EersteProg
            then Omgeving := ActPSP^.Omgeving
            else Omgeving := 0;
         if DitSegment + 1 = Eigenaar then begin
            ProgGevonden  := TRUE;
            Functie       := 'Prog';
            KorteNaam[0]  := #0;
            while (ActMCB^.Naam[ ord (KorteNaam[0]) + 1 ] <> #0) and
                  (KorteNaam[0] < #8) do
            begin
               inc (KorteNaam[0]);
               KorteNaam[ ord (KorteNaam[0]) ] :=
                  ActMCB^.Naam[ ord (KorteNaam[0]) ];
            end;
            if Eigenaar = prefixseg then begin
               TerugkeerSegment := word (ptr (prefixseg, $000C)^);
               TerugkeerOffset  := word (ptr (prefixseg, $000A)^);
               LangeNaam        := '-----> Terminate Vector = '     +
                                   WordHex (TerugkeerSegment) + ':' +
                                   WordHex (TerugkeerOffset )        ;
            end
            else
               LangeNaam := '';
         end {if ÆProgØ}
         else begin
            if Eigenaar = $0008 then begin
               if ActMCB^.Naam[1] = 'S' then
                  case ActMCB^.Naam[2] of
                     'D' : Functie := 'SysD';
                     'C' : Functie := 'SysP';
                     else  Functie := 'Data';
                  end {case}
               else        Functie := 'Data';
               KorteNaam := '';
               LangeNaam := '';
            end {if Eigenaar = $0008}
            else begin
               if DitSegment + 1 = Omgeving then begin
                  Functie   := 'Env ';
                  LangeNaam := Coda (Omgeving, Paragrafen);
                  if EersteProg = Eigenaar then Meester := Omgeving;
               end {if ÆEnvØ}
               else begin
                  move (ptr (DitSegment + 1, 0)^, Specificatie[1], 8);
                  if (Specificatie = 'PATH=' + #0 + 'CO') or
                     (Specificatie = 'COMSPEC='         ) or
                     (Specificatie = 'OS=DRDOS'         ) then
                  begin
                     Functie   := 'Env' + chr (39);
                     LangeNaam := Coda (DitSegment + 1, Paragrafen);
                     if (EersteProg = Eigenaar) and
                        (Meester    = $0000   )
                     then
                        Meester := DitSegment + 1;
                  end
                  else begin
                     if Eigenaar = 0
                        then Functie := 'Free'
                        else Functie := 'Data';
                     LangeNaam := '';
                     if (EersteProg = Eigenaar) and
                        (Meester    = $0000   )
                     then
                        Meester := DitSegment + 1;
                  end;
               end {else: not ÆEnvØ};
               KorteNaam := '';
            end {else: Eigenaar <> $0008};
         end {else: not ÆProgØ};

        {----- KorteNaam redigeren -----}
         for i := 1 to length (KorteNaam) do
            if KorteNaam[i] < #32 then KorteNaam[i] := '.';
         KorteNaam := KorteNaam + '        ';

        {----- Oorsprong vaststellen -----}
         if EersteProg = Eigenaar
            then Oorsprong := '*'
            else Oorsprong := ' ';

        {----- Actueel proces (uitgaande Pijl) vaststellen -----}
         if Eigenaar = prefixseg
            then Pijl := '>'
            else Pijl := ' ';
      end {with Schakel^};

     {----- MCB-opeenvolging onderzoeken / schakelverloop vaststellen -----}
      if (Zone = 'Upper') and (ActMCB^.Signatuur = 'Z') then begin
         Schakel^.Volgende := nil;
         EindeKeten        := TRUE;
      end
      else begin
         ActMCB := ptr (seg (ActMCB^) + ActMCB^.Paragrafen + 1, 0);
         if ((ActMCB^.Signatuur <> 'M') and (ActMCB^.Signatuur <> 'Z')) or
            ($FFFF - ActMCB^.Paragrafen < seg (ActMCB^)               )
         then begin
            Schakel^.Volgende := nil;
            EindeKeten        := TRUE;
         end
         else begin
            new (LaatsteSchakel);
            Schakel^.Volgende      := LaatsteSchakel;
            LaatsteSchakel^.Vorige := Schakel;
            Schakel                := LaatsteSchakel;
         end {else: (ÆMØ or ÆZØ) and Æteveel_ParagrafenØ};
      end {else: ÆLowerØ or not ÆZØ};
   until EindeKeten;

  {----- Terugtocht -----}
   while Schakel <> nil do with Schakel^ do begin

     {----- Ouder-proces vaststellen -----}
      TerugkeerSegment2 := TerugkeerSegment + (TerugkeerOffset div $10);
      if (DitSegment              <= TerugkeerSegment2) and
         (DitSegment + Paragrafen >= TerugkeerSegment2)
      then
         OuderSegment := Eigenaar;

     {----- Meester-omgeving markeren -----}
      if DitSegment + 1 = Meester then Oorsprong := 'M';

     {----- Schakel-verloop -----}
      Schakel := Schakel^.Vorige;
   end {while Schakel <> nil};

  {----- Rapportage -----}
   writeln ('Chain of Memory Control Blocks in DOS version ',
            lo (dosversion), '.', ReleaseNummer, ':');
   writeln;
   writeln ('MCB@ #Par Signat PSP@ Env@ Type !! Name     File');
   writeln ('---- ---- ------ ---- ---- ---- -- -------- ',
            '-----------------------------------');
   Schakel := EersteSchakel;
   while Schakel <> nil do with Schakel^ do begin

     {----- Ouder-omgeving vaststellen -----}
      if Eigenaar = OuderSegment then begin
         if not Dos3punt2 then begin
            if (Functie = 'Env ') then begin
               Ouder := DitSegment + 1;
               Pijl  := 'Û';
            end
            else
               Pijl := '<';
         end {if not Dos3punt2}
         else begin
            if ((Functie = 'Env' + chr (39)) or (Functie = 'Data')) and
               (Ouder    = $0000)
            then begin
               Ouder := DitSegment + 1;
               Pijl  := 'Û';
            end
            else
               Pijl := '<';
         end {else: Dos3punt2};
      end {with Schakel^};

     {----- Keten-weergave -----}
      writeln (WordHex (DitSegment)        , ' ',
               WordHex (Paragrafen)        , ' ',
               Signatuur                   , ' ',
               WordHex (Eigenaar)          , ' ',
               WordHex (Omgeving)          , ' ',
               Functie                     , ' ',
               Oorsprong, Pijl             , ' ',
               KorteNaam                   , ' ',
               LangeNaam                        );

     {----- Schakel-verloop -----}
      Schakel := Schakel^.Volgende;
   end {while Schakel <> nil};

  {----- Afsluiting rapportage -----}
   writeln;

   write ('* = First command interpreter at ');
   if ProgGevonden
      then writeln (WordHex (EersteProg), ':0000')
      else writeln ('?');

   write ('M = Master environment        at ');
   if Meester > $0000
      then writeln (WordHex (Meester), ':0000')
      else writeln ('?');

   write ('< = Parent proces             at ');
   writeln (WordHex (OuderSegment), ':0000');

   write ('Û = Parent environment        at ');
   if Ouder > $0000
      then writeln (WordHex (Ouder), ':0000')
      else writeln ('?');

   writeln ('> = Current proces            at ',
            WordHex (prefixseg), ':0000');

   writeln ('    returns                   to ',
            WordHex (TerugkeerSegment), ':', WordHex (TerugkeerOffset));
END.

(Above ASCII 127, there may be some ASCII/ANSI translation issues in this presentation.)

Answer 3

Using setx has few drawbacks, especially if you're trying to append to environment variables (eg. setx PATH %Path%;C:\mypath) This will repeatedly append to the path every time you run it, which can be a problem. Worse, it doesn't distinguish between the machine path (stored in HKEY_LOCAL_MACHINE), and the user path, (stored in HKEY_CURRENT_USER). The environment variable you see at a command prompt is made up of a concatenation of these two values. Hence, before calling setx:

user PATH == u
machine PATH == m
%PATH% == m;u

> setx PATH %PATH%;new

Calling setx sets the USER path by default, hence now:
user PATH == m;u;new
machine PATH == m
%PATH% == m;m;u;new

The system path is unavoidably duplicated in the %PATH% environment variable every time you call setx to append to PATH. These changes are permanent, never reset by reboots, and so accumulate through the life of the machine.

Trying to compensate for this in DOS is beyond my ability. So I turned to Python. The solution I have come up with today, to set environment variables by tweaking the registry, including appending to PATH without introducing duplicates, is as follows:

from os import system, environ
import win32con
from win32gui import SendMessage
from _winreg import (
    CloseKey, OpenKey, QueryValueEx, SetValueEx,
    HKEY_CURRENT_USER, HKEY_LOCAL_MACHINE,
    KEY_ALL_ACCESS, KEY_READ, REG_EXPAND_SZ, REG_SZ
)

def env_keys(user=True):
    if user:
        root = HKEY_CURRENT_USER
        subkey = 'Environment'
    else:
        root = HKEY_LOCAL_MACHINE
        subkey = r'SYSTEM\CurrentControlSet\Control\Session Manager\Environment'
    return root, subkey


def get_env(name, user=True):
    root, subkey = env_keys(user)
    key = OpenKey(root, subkey, 0, KEY_READ)
    try:
        value, _ = QueryValueEx(key, name)
    except WindowsError:
        return ''
    return value


def set_env(name, value):
    key = OpenKey(HKEY_CURRENT_USER, 'Environment', 0, KEY_ALL_ACCESS)
    SetValueEx(key, name, 0, REG_EXPAND_SZ, value)
    CloseKey(key)
    SendMessage(
        win32con.HWND_BROADCAST, win32con.WM_SETTINGCHANGE, 0, 'Environment')


def remove(paths, value):
    while value in paths:
        paths.remove(value)


def unique(paths):
    unique = []
    for value in paths:
        if value not in unique:
            unique.append(value)
    return unique


def prepend_env(name, values):
    for value in values:
        paths = get_env(name).split(';')
        remove(paths, '')
        paths = unique(paths)
        remove(paths, value)
        paths.insert(0, value)
        set_env(name, ';'.join(paths))


def prepend_env_pathext(values):
    prepend_env('PathExt_User', values)
    pathext = ';'.join([
        get_env('PathExt_User'),
        get_env('PathExt', user=False)
    ])
    set_env('PathExt', pathext)



set_env('Home', '%HomeDrive%%HomePath%')
set_env('Docs', '%HomeDrive%%HomePath%\docs')
set_env('Prompt', '$P$_$G$S')

prepend_env('Path', [
    r'%SystemDrive%\cygwin\bin', # Add cygwin binaries to path
    r'%HomeDrive%%HomePath%\bin', # shortcuts and 'pass-through' bat files
    r'%HomeDrive%%HomePath%\docs\bin\mswin', # copies of standalone executables
])

# allow running of these filetypes without having to type the extension
prepend_env_pathext(['.lnk', '.exe.lnk', '.py'])

It does not affect the current process or the parent shell, but it will affect all cmd windows opened after it is run, without needing a reboot, and can safely be edited and re-run many times without introducing any duplicates.

Answer 4

It may be just as easy to use the external Windows setx command:

C:\>set NEWVAR
Environment variable NEWVAR not defined

C:\>python
Python 2.5.4 (r254:67916, Dec 23 2008, 15:10:54) [MSC v.1310 32 bit (Intel)] on
win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.system('setx NEWVAR newvalue')
0
>>> os.getenv('NEWVAR')
>>> ^Z


C:\>set NEWVAR
Environment variable NEWVAR not defined

Now open a new Command Prompt:

C:\>set NEWVAR
NEWVAR=newvalue

As you can see, setx neither sets the variable for the current session, nor for the parent process (the first Command Prompt). But it does set the variable persistently in the registry for future processes.

I don't think there is a way of changing the parent process's environment at all (and if there is, I'd love to hear it!).

Answer 5

In the os module, there is getenv and putenv functions. However, it seems that the putenv is not working correctly and that you must use the windows registry instead

Look at this discussion

Answer 6

The registry way is if you want to modify it permanently for everything, which I guess is what you want here since it's in setup.py.

Temporarily for just your process, then os.environ is the trick.