Since we have a pre-initialized logger, I think we should avoid using print() command.

No problem, updated.

I'm not sure this check is enough..., maybe the upload_build_results is clever enough to handle issues? But see how this checking is done for the resultdir cleaner crawler.

You are right, that would cause problems. Updated.

I suppose we can not make this in a transactional manner.. (if error happens, rollback). But would it be possible to first analyze the situation and gather the tasks that need to be done, fail if some problem happens, and only if no problems happen - start the processing?

Also, I'm curious if whether we need a project lock (for building and other modification).

But would it be possible to first analyze the situation and gather the tasks that need to be done, fail if some problem happens, and only if no problems happen - start the processing?

Sooo, I am not really sure how helpful this would be. Gathering tasks beforehand would probably avoid issues like the script trying to access a directory it doesn't have permissions to and then failing. Or something like this. But I suppose the majority of failures that can/will happen are going to happen due to networking issues or something else when actually uploading things to Pulp. And having a calculated list of tasks wouldn't IMHO help.

I would probably only remember or maybe pre-calculate the number of RPM files we are uploading and after everything is done, query Pulp to find out if we have the same number. Or maybe compare names of RPMs if we wanted to be more precise. If it doesn't match, we can either re-try several times, or just log it and manually review all failures.

Also, I'm curious if whether we need a project lock (for building and other modification).

That dumping a lockfile in this script would be easy but changing our build-related code, action code, cron jobs, etc to respect the lock, sounds like a bigger problem.

If such a locking feature would be generally useful, then sure. But if the only purpose would be for the Pulp migration, I hope we could figure something easier.

For initial migrations of test users, I think we would be fine with "please don't submit new builds until the migration is finished". And the mass migration of everything, will be done in batches. So maybe we can just put an ugly hack into our build/action scheduler to temporarily hide all jobs that fall in the currently migrated batch.