Django custom form with non-model fields in Admin/AdminInline

Question

I'm having a bit of trouble with customizing the Django admin pages to be able to use custom forms or non-model fields.

The site I'm working on will be used as a data storage utility for reporting on some manufacturing processes. The database will have three main tables, two of which will have a ForeignKey in a fourth table. Each row in the fourth table is essentially a single line from a CSV file that needs to be uploaded. The model for the fourth table looks like this:

bandwidthrawid = models.IntegerField(...) 
testdate = models.DateTimeField(...)
frequency = models.FloatField(...)
power = models.FloatField(...)
uncalibratedpower = models.FloatField(...)

I'm letting Django automatically generate the id field. The test date will be static for every entry from the same test. The Frequency, Power, and UncalibratedPower fields will all contain a single value from the same line of the CSV file. The BandwidthRawId field will contain integers in sequential order, many times over again indicating which specific test this was. So, for example, let's assume a user tests the same module twice, over a frequency range of 1 to 60 Hz in steps of 1 Hz, a CSV file will be generated that looks similar to this:

1,0,0
2,0,0
3,0,0
4,0,0
5,0,0
...
60,0,0

The second test will also generate a similar file. When these files are uploaded into the system, the date of the test will be recorded, and it will be assigned a BandwidthRawId which is an incremented value of the current highest value in that column. So, the table entry for the two files will look like this:

Id, Date, BWID, F, P, UP
1,  Date1, 1,   1, 0, 0
2,  Date1, 1,   2, 0, 0
...
60,  Date1, 1,   60, 0, 0
61,  Date2, 2,   1, 0, 0
62,  Date2, 2,   2, 0, 0
...
120,  Date2, 2,   60, 0, 0

This is where things start to get tricky. Sometimes the files are not all in the same order, so the CSV Importer won't work. Sometimes the files don't contain all of the right information. The files NEVER have a date in them, and NEVER have any header lines. In order to combat all of these things, I created a widget/field that creates a CSV Preview and allows the user to select which actual field each column belongs to. I called this CSVPreviewField.

In order to test this field/widget in the "field" I overrode the BandwidthRawID field in the model and was using that to handle all of the processing. Originally I thought it would be easy to override that field, do the processing, and then fill in that field with the correct integer, then pass that off for processing. Unfortunately, that doesn't work. I get the error that the BandwidthRawID field is of invalid type (in the database it's an integer, in the model it's CSVPreviewField/FileInput).

Next, I tried to add a CSVPreviewField (called it BandwidthRawFile) to a ModelForm that uses BandwidthRaw as a model and overrode the form field of the ModelAdmin class I wrote to handle the bandwidth_raw table. Unfortunately, no matter what I do, I can't display this field, the error I get is "BandwidthRawFile is not found in form fields" or something similar. I've also gotten an error which indicates that there's no column in the database that corresponds with "BandwidthRawFile."

After that, I learned about inlines, and attempted the same thing with an inline form, which also fails for similar reasons. Either it fails because I don't specify a model (was hoping I could do a custom form), the model doesn't contain the BandwidthRawFile field, or the database doesn't contain the field.

At this point, I've been working on this for two days and have run out of ideas completely. Basically, the inline form would be the best solution. If I could some-how put my widget inline in the other two admin pages that require the BandwidthRawId and then return that value as a result of the widget processing, that would be the ideal situation. Right now I'd be happy just to have the little green + next to the foreign key launch a custom form that does all the processing and returns the BandwidthRawId.

Answer 1

Since it looks like nobody has an answer to this, I'm going to go ahead and "answer" it.

The answer I've found is that you cannot have non-model fields in a model-form. They are simply ignored by the framework. Adding your own forms inline also doesn't seem to work well, the same errors end up occurring.

The way I fixed this issue was:

1. Create A CSV Preview Widget
2. Implement the Widget into a CSV Field
3. Write the field/widget logic
4. Create my own form -- NOT a model-form
5. Create Form handlers
6. Create view to host the form
7. Override add_view in models to return the custom form
8. Override the get_form function in models to add the csv widget to the form

I don't really -Like- this answer, but I don't have any better solution.

Answer 2

If you are not too picky on the implementation you might want to look at something like Postgres Foreign data wrappers (file_fdw is a csv interface). Since the files are very uniform, you will get a nice ORM interface and save a ton of headaches on the import side.