Module organization, inheritance, and @classmethods

Question

I'm trying to write a class that works kind of like the builtins and some of the other "grown-up" Python stuff I've seen. My Pythonic education is a little spotty, classes-wise, and I'm worried I've got it all mixed up.

I'd like to create a class that serves as a kind of repository, containing a dictionary of unprocessed files (and their names), and a dictionary of processed files (and their names). I'd like to implement some other (sub?)classes that handle things like opening and processing the files. The file handling classes should be able to update the dictionaries in the main class. I'd also like to be able to directly call the various submodules without having to separately instantiate everything, e.g.:

import Pythia
p = Pythia()
p.FileManager.addFile("/path/to/some/file")

or even

Pythia.FileManager.addFile("/path/to/some/file")

I've been looking around at stuff about @classmethod and super and such, but I can't say I entirely understand it. I'm also beginning to suspect that I might have the whole chain of inheritance backwards--that what I think of as my main class should actually be the child class of the handling and processing classes. I'm also wondering whether this would all work better as a package, but that's a separate, very intimidating issue.

Here's my code so far:

#!/usr/bin/python

import re
import os
class Pythia(object):
    def __init__(self):
        self.raw_files = {}
        self.parsed_files = {}
        self.FileManger = FileManager()
    def listf(self,fname,f):
        if fname in self.raw_files.keys():
            _isRaw = "raw"
        elif fname in self.parsed_files.keys():
            _isRaw = "parsed"
        else:
            return "Error: invalid file"
        print "{} ({}):{}...".format(fname,_isRaw,f[:100])

    def listRaw(self,n=None):
        max = n or len(self.raw_files.items())
        for item in self.raw_files.items()[:max]:
            listf(item[0],item[1])

    def listParsed(self,n=None):
        max = n or len(self.parsed_files.items())
        for item in self.parsed_files.items()[:max]:
            listf(item[0],item[1])

class FileManager(Pythia):
    def __init__(self):
        pass
    def addFile(self,f,name=None,recurse=True,*args):
        if name:
            fname = name
        else:
            fname = ".".join(os.path.basename(f).split(".")[:-1])
        if os.path.exists(f):
            if not os.path.isdir(f):
                with open(f) as fil:
                    Pythia.raw_files[fname] = fil.read()
            else:
                print "{} seems to be a directory.".format(f)
                if recurse == False:
                    return "Stopping..."
                elif recurse == True:
                    print "Recursively navingating directory {}".format(f)
                    addFiles(dir,*args)
                else:
                    recurse = raw_input("Recursively navigate through directory {}? (Y/n)".format(f))
                    if recurse[0].lower() == "n":
                        return "Stopping..."
                    else:
                        addFiles(dir,*args)
        else:
            print "Error: file or directory not found at {}".format(f)
    def addFiles(self,directory=None,*args):
        if directory:
            self._recursivelyOpen(directory)
        def argHandler(arg):
            if isinstance(arg,str):
                self._recursivelyOpen(arg)
            elif isinstance(arg,tuple):
                self.addFile(arg[0],arg[1])
            else:
                print "Warning: {} is not a valid argument...skipping..."
                pass
        for arg in args:
            if not isinstance(arg,(str,dict)):
                if len(arg) > 2:
                    for subArg in arg:
                        argHandler(subArg)
                else:
                    argHandler(arg)
            elif isinstance(arg,dict):
                for item in arg.items():
                    argHandler(item)
            else:
                argHandler(arg)
    def _recursivelyOpen(self,f):
        if os.path.isdir(f):
            l = [os.path.join(f,x) for x in os.listdir(f) if x[0] != "."]
            for x in l:
                _recursivelyOpen(x)
        else:
            addFile(f)

Answer 1

There is a lot here and you are probably best to educate yourself some more.

For your intended usage:

import Pythia
p = Pythia()
p.file_manager.addFile("/path/to/some/file")

A class structure like this would work:

class FileManager(object):
    def __init__(self, parent):
        self.parent = parent

    def addFile(self, file):
        # Your code
        self.parent.raw_files[file] = file

    def addFiles(self, files)
        # Your code
        for file in files:
            self.parent.raw_files[file] = file

class Pythia(object):
    def __init__(self):
        self.file_manager = FileManager(self)

However there are a lot of options. You should write some client code first to work out what you want, then implement your class/objects to match that. I don't tend to ever use inheritance in python, it is not really required due to pythons duck typing.

Also if you want a method to be called without instantiating the class use staticmethod, not classmethod. For example:

class FileManager(object):
    @staticmethod
    def addFiles(files):
        pass

Answer 2

First off: follow PEP8's guidelines. Module names, variable names, and function names should be lowercase_with_underscores; only class names should be CamelCase. Following your code is a little difficult otherwise. :)

You're muddying up OO concepts here: you have a parent class that contains an instance of a subclass.

Does a FileManager do mostly what a Pythia does, with some modifications or extensions? Given that the two only work together, I'd guess not.

I'm not quite sure what you ultimately want this to look like, but I don't think you need inheritance at all. FileManager can be its own class, self.file_manager on a Pythia instance can be an instance of FileManager, and then Pythia can delegate to it if necessary. That's not far from how you're using this code already.

Build small, independent pieces, then worry about how to plug them into each other.

---

Also, some bugs and style nits:

• You call _recursivelyOpen(x) but forgot the self..

• Single space after commas.

• Watch out for max as a variable name: it's also the name of a builtin function.

• Avoid type-checking (isinstance) if you can help it. It's extra-hard to follow your code when it does a dozen different things depending on argument types. Have very clear argument types, and create helper functions that accept different arguments if necessary.

• You have Pythia.raw_files[fname] inside FileManager, but Pythia is a class, and it doesn't have a raw_files attribute anyway.

• You check if recurse is True, then False, then... something else. When is it something else? Also, you should use is instead of == for testing against the builtin singletons like this.