Page 1 of 1

Manual Import

PostPosted: Thu Jan 24, 2008 10:39 pm
by ozora tsubasa
I'm currently importing around 5000 files worth of handhistories. The import crashed once around file 750. I moved the file to another folder, restarted the import. This time the 1000th file or so crashed the import, and it takes around 15 minutes for PT3 to read/skip already imported files to reach 1000th file. So once again, I removed the problem file and restarted. Eventually I'm sure I'll locate all files causing the crashes but you probably get my point. Is there a way to move processed files along the way? I have it checked on auto-import tab, but it doesn't seem to be doing anything for the manual import. Or maybe it is waiting until all files are imported.

Is there a way I can do this less painfully?

Re: Manual Import

PostPosted: Thu Jan 24, 2008 10:48 pm
by _dave_
[quote2x4]
Is there a way to move processed files along the way?
[/quote2x4]
sounds like a very good idea imo.

Re: Manual Import

PostPosted: Thu Jan 24, 2008 11:38 pm
by APerfect10
[quoteasi]Is there a way I can do this less painfully?[/quoteasi]

Not currently. Once PT3 stabilizes and it is not crashing, will the move processed files be useful? Ideally, you import them once and PT3 will never have to re-read the directory.

The reason for moving processed files for auto-import is because PT3 is constantly reading then re-reading the directory, over and over again.

Best regards,

Derek

Re: Manual Import

PostPosted: Thu Jan 24, 2008 11:54 pm
by ozora tsubasa
[quote="APerfect10"7jw]
Not currently. Once PT3 stabilizes and it is not crashing, will the move processed files be useful? Ideally, you import them once and PT3 will never have to re-read the directory.
Derek[/quote7jw]

It won't be useful once PT3's not crashing.
I split my files into batches of 800 into a few folders until then as a solution and so far so good.

One thing might help others though. In what order is PT3 processing files. Name, date, something else? If I knew this, I could just sort files and move the ones before the problem file knowing they would have been processed.

Re: Manual Import

PostPosted: Thu Jan 24, 2008 11:56 pm
by _dave_
yeah - my "very good idea" point was really meant as an aid to finding problem files at this debugging stage.

Agree it willbe pointless once import is bug-free :) (well, unless you have a power failure / crashy CPU lol)

Re: Manual Import

PostPosted: Thu Jan 24, 2008 11:57 pm
by APerfect10
Good question and I am not sure. Josh will be able to clarify...

-AP10

Re: Manual Import

PostPosted: Fri Jan 25, 2008 1:47 am
by ozora tsubasa
[quote="APerfect10"byb]Good question and I am not sure. Josh will be able to clarify...

-AP10[/quotebyb]

It looks like the import is done by filename. So, sorting by name and removing the files upto the problem file does the trick.