Migration failed with pg_restore: [parallel archiver] could not create worker process: Cannot allocate memory

[tanyaranjan1995]

Migration was stuck in restore of pre-data section to the target database for 30+ hours.
Cx is trying to migrate 1 database of size 83 GB with 27 tables and 29 indexes from from sspg 11 to fspg 11.

Error Logs:

PgcopydbCloneAction syncop 2024-07-30 12:49:48.059 82 INFO cli_clone_follow.c:591   STEP 3: restore the pre-data section to the target database
PgcopydbCloneAction syncop 2024-07-30 12:49:48.065 82 SQL  pgsql.c:502              Connecting to [target] "postgres://adminsyncop@localhost:5432/syncop?sslmode=prefer&keepalives=1&keepalives_idle=10&keepalives_interval=10&keepalives_count=60"
PgcopydbCloneAction syncop 2024-07-30 12:49:48.071 82 SQL  pgsql.c:1569             [TARGET 20675] BEGIN;
PgcopydbCloneAction syncop 2024-07-30 12:49:48.072 82 SQL  pgsql.c:1569             [TARGET 20675] COMMIT;
PgcopydbCloneAction syncop 2024-07-30 12:49:48.072 82 SQL  pgsql.c:411              Disconnecting from [target] "postgres://adminsyncop@localhost:5432/syncop?sslmode=prefer&keepalives=1&keepalives_idle=10&keepalives_interval=10&keepalives_count=60"
PgcopydbCloneAction syncop 2024-07-30 12:49:48.072 82 NOTICE pgcmd.c:1046              /usr/lib/postgresql/11/bin/pg_restore -f /datadrive/migrations_data/1bc882bd-bb77-408d-849c-536b9b22771e/oid-16501/schema/pre-out.list -l /datadrive/migrations_data/1bc882bd-bb77-408d-849c-536b9b22771e/oid-16501/schema/schema.dump
PgcopydbCloneAction syncop 2024-07-30 12:57:46.338 82 INFO pgcmd.c:989               /usr/lib/postgresql/11/bin/pg_restore --dbname 'postgres://adminsyncop@localhost:5432/syncop?sslmode=prefer&keepalives=1&keepalives_idle=10&keepalives_interval=10&keepalives_count=60' --section pre-data --jobs 48 --no-tablespaces --use-list /datadrive/migrations_data/1bc882bd-bb77-408d-849c-536b9b22771e/oid-16501/schema/pre-filtered.list /datadrive/migrations_data/1bc882bd-bb77-408d-849c-536b9b22771e/oid-16501/schema/schema.dump
PgcopydbCloneAction syncop 2024-07-31 19:05:25.905 82 ERROR string_utils.c:682       pg_restore: [parallel archiver] could not create worker process: Cannot allocate memory
PgcopydbCloneAction syncop 2024-07-31 19:05:26.703 82 ERROR pgcmd.c:1003             Failed to run pg_restore: exit code 1
PgcopydbCloneAction syncop 2024-07-31 19:05:26.703 82 ERROR cli_clone_follow.c:595   Failed to prepare schema on the target database, see above for details
PgcopydbCloneAction syncop 2024-07-31 19:05:26.703 82 ERROR cli_clone_follow.c:503   Failed to clone source database, see above for details
PgcopydbCloneAction syncop 2024-07-31 19:05:26.815 63 ERROR cli_clone_follow.c:763   clone process 82 has terminated [6]

[dimitri]

Given that the problem is from pg_restore, can you reproduce the problem just using this tool, without pgcopydb involved?

[tanyaranjan1995]

@dimitri I was able to restore the schema without pgcopydb that I tried locally. Didnt face any issue wrt to the error.