CODEWITHSUNDEEP
pythonperforcep4python
Xolin_
Xolin_

Reputation: 97

P4python how do I mark a list of files for add/edit?

I have a TON of files (~260,000) sorted into files with lists of files by type. (binary, binary+Dx, text, text+x, etc...)

EI -

binary:

D:/Workspace/folder1/file.dll
D:/Workspace/folder2/file.res
D:/Workspace/folder3/file.pyd

text:

D:/Workspace/folder1/file.txt
D:/Workspace/folder2/file.txt
D:/Workspace/folder3/file.doc

etc, etc....

Using p4python, is there a way to add a list of files to a changelist? I have managed to find a way to add these line-by-line, but that will take FOREVER; I think I calculated it at like 3 days.... This method takes about 1 sec per line:

for lines in fileList:
    p4.run_edit(f"-c{changeNum}", f"-t{fileName}", lines )

The command I would try to mimic is like:

p4 add -t text -c 1214 D:/Workspace/folder1/file.txt, D:/Workspace/folder2/file.txt, D:/Workspace/folder3/file.doc

The documentation for p4python suggests that there is a way to use files that are already marked for add (in the default CL?), but that still requires these files to be added.

I have also found the "-x" option for add Here, but p4python doesn't seem to like this option. At least I can't get it to accept it...

In a nut-shell: I am just looking for a relatively time-efficient way to get the files from my disk to the depot, 100% in-code. My knowledge of Python is still pretty limited, so please forgive me if there is a super-obvious way to do this that I don't know about.

Here is what I have so far....

def generate_changelist( fileLoc, fileType ):
    fileList = []
    ## starts 'New' changelist
    print("Generating Changelist...")
    change = p4.fetch_change()
    change._description = f"P4 Script; Uploading files of type: {fileType}"
    
    ## give it a Number
    CL = p4.save_change( change ) # returns a list with the string "Change ### created"
    print(CL)
    for item in CL: changeNum = item.split()[1]
    
    print(f"Attaching Files to CL {changeNum}")
    with open(fileLoc, 'r') as f:
        for line in f:
            line = line[:-1]    # trims the '\n' off the end...
            fileList.append(line) # p4 command would go here....
    ## tinkering with these right now....
    p4.run_edit(f"-c{changeNum}", f"-t{fileType}", f"{fileList}")
    #p4.run("-s", f"-x {fileList}", "add" )
    
    ## works....but painfully slow....
    ## I know I can add this to the for loop above...
    #for lines in fileList:
    #    p4.run_edit(f"-c{changeNum}", f"-t{fileType}", f"{lines}")
    return changeNum

EDIT / Solution-ish: SO i was able to get this to work, kinda:

print(f"Attaching Files to CL {changeNum}")
with open(fileLoc, 'r') as f:
    for line in f:
        line = line[:-1]    # trims the '\n' off the end...
        if (DEBUG): print(line)
        fileList.append(line)
p4.run_edit(f"-c{changeNum}", f"-t{fileName}", fileList)

There seems to be an issue that P4Python is not throwing the error:

Screenshot

So, I think it might be that the list is just too large. Might need to have a counter to break up the fileList into chunks... 5k per?

Thanks all for the help!!

Upvotes: 0

Views: 380

Answers (2)

Samwise
Samwise

Reputation: 71424

Focusing on the "in a nutshell" part:

I am just looking for a relatively time-efficient way to get the files from my disk to the depot, 100% in-code.

Generating big lists of files and then using them to drive a script is certainly possible, but it's significantly less efficient than doing it the conventional way, and smacks of "XY problem".

Getting files from your disk to the depot in a Python script should usually be as simple as:

p4.run_reconcile()
p4.run_submit("-d", "P4 script uploading files")

If you want to set custom filetypes based on extension, this is best done via the p4 typemap command; the typemap is persistently stored on the server and is automatically applied when you add a file. A file's type "sticks" with it unless explicitly changed, so it's not necessary to re-specify it every time you make an edit.

Upvotes: 0

Barmar
Barmar

Reputation: 780688

Don't put filelist in a single argument, they should be separate arguments.

p4.run_edit(f"-c{changeNum}", f"-t{fileType}", *fileList)

Upvotes: 2

Related Questions