Python Scapy and Pyinstaller

Question

I'm trying to create simple executable with pyinstaller from script I use for testing so I do not have to install everything on server I'm testing from.

#! /usr/bin/env python

from scapy.all import *

sourceport=int(raw_input('Soruce port:'))
destinationport=int(raw_input('Destination port:'))
destinationip=raw_input('Destination IP:')
maxttl=int(raw_input('MAX TTL:'))

for i in range(1,maxttl):
     udptrace = IP(dst=destinationip,ttl=i)/UDP(dport=destinationport,sport=sourceport,len=500)
     received=sr1(udptrace,verbose=0,timeout=2)
     try:
         print received.summary()
     except AttributeError:
         print "** TIMEOUT **"

Then I make executable:

pyinstaller -F udp.py

However when I run it and I have the following error:

Soruce port:500
Destination port:500
Destination IP:4.4.4.4
MAX TTL:3
Traceback (most recent call last):
  File "<string>", line 16, in <module>
NameError: name 'IP' is not defined
user@:~/2/dist$

I have spent some time researching but did not find any answers.

Answer 1

The problem

First, we need to pinpoint the exact problem.

The PyInstaller manual specifies that:

Some Python scripts import modules in ways that PyInstaller cannot detect: for example, by using the __import__() function with variable data.

Inspecting Scapy's source code reveals that this is exactly how the various networking layers are imported:

1. scapy/layers/all.py:

   def _import_star(m):
       mod = __import__(m, globals(), locals())
       for k,v in mod.__dict__.iteritems():
           globals()[k] = v
   
   for _l in conf.load_layers:
       log_loading.debug("Loading layer %s" % _l)
       try:
           _import_star(_l)
       except Exception,e:
       log.warning("can't import layer %s: %s" % (_l,e))
   

   Note that __import__ is invoked for each module in conf.load_layers.

2. scapy/config.py:

   class Conf(ConfClass):
       """This object contains the configuration of scapy."""
        load_layers = ["l2", "inet", "dhcp", "dns", "dot11", "gprs", "hsrp", "inet6", "ir", "isakmp", "l2tp",
                      "mgcp", "mobileip", "netbios", "netflow", "ntp", "ppp", "radius", "rip", "rtp",
                      "sebek", "skinny", "smb", "snmp", "tftp", "x509", "bluetooth", "dhcp6", "llmnr", "sctp", "vrrp",
                      "ipsec" ]
   

   Note that Conf.load_layers contains "inet".

3. The file scapy/layers/inet.py defines the IP class, which was not imported successfully in the enclosed example.

The solution

Now, that we have located the root cause, let's see what can be done about it.

The PyInstaller manual suggests some workarounds to such importing issues:

• You can give additional files on the PyInstaller command line.
• You can give additional import paths on the command line.
• You can edit the myscript.spec file that PyInstaller writes the first time you run it for your script. In the spec file you can tell PyInstaller about code and data files that are unique to your script.
• You can write "hook" files that inform PyInstaller of hidden imports. If you "hook" imports for a package that other users might also use, you can contribute your hook file to PyInstaller.

A bit of googling reveals that an appropriate "hook" was already added to the default PyInstaller distribution, in this commit, which introduced the file PyInstaller/hooks/hook-scapy.layers.all.py.

The PyInstaller manuals indicates that such built-in hooks should run automatically:

In summary, a "hook" file tells PyInstaller about hidden imports called by a particular module. The name of the hook file is hook-<module>.py where "<module>" is the name of a script or imported module that will be found by Analysis. You should browse through the existing hooks in the hooks folder of the PyInstaller distribution folder, if only to see the names of the many supported imports.

For example hook-cPickle.py is a hook file telling about hidden imports used by the module cPickle. When your script has import cPickle the Analysis will note it and check for a hook file hook-cPickle.py.

Bottom Line

Therefore, please verify that you're running the latest version of PyInstaller. If you can't upgrade to the latest version, or if it doesn't contain the file PyInstaller/hooks/hook-scapy.layers.all.py, then create it with the following content:

#-----------------------------------------------------------------------------
# Copyright (c) 2013, PyInstaller Development Team.
#
# Distributed under the terms of the GNU General Public License with exception
# for distributing bootloader.
#
# The full license is in the file COPYING.txt, distributed with this software.
#-----------------------------------------------------------------------------

from PyInstaller.hooks.hookutils import collect_submodules

# The layers to load can be configured using scapy's conf.load_layers.
#  from scapy.config import conf; print(conf.load_layers)
# I decided not to use this, but to include all layer modules. The
# reason is: When building the package, load_layers may not include
# all the layer modules the program will use later.

hiddenimports = collect_submodules('scapy.layers')