How does cbcopy handle a duplicate value conflict when the DISTRIBUTED BY column is also the PRIMARY KEY?

[asheraz048i2c]

Context:
I'm migrating a production database from Greenplum 7 to Apache Cloudberry 2.0.1 using cbcopy. One of the tables (~880 million rows) has its DISTRIBUTED BY column also defined as the PRIMARY KEY.
Question:
If the source table contains duplicate values in this column — for example due to a constraint that was disabled or marked NOT VALID at some point on the source side — what is cbcopy's actual behavior when loading into a target table where the PRIMARY KEY constraint is active?
Specifically:
Does cbcopy do any pre-check on this column before starting the load, or does it just attempt the insert and let the target's constraint layer catch it?
When a duplicate key violation happens mid-load, does the entire table/partition load abort and roll back, or does it skip the conflicting row(s) and continue with the rest?
Is there any flag or option to control this behavior — for example, skipping and logging conflicting rows versus a hard failure?
I'd like a definitive answer before running this on a table this large in production, rather than relying on trial-and-error. Any insight from someone who has hit this scenario during a large-scale migration would help a lot.
Environment:
Source: Greenplum 7.x
Target: Apache Cloudberry 2.0.1
Table size: ~880 million rows
Column: DISTRIBUTED BY + PRIMARY KEY (same column)

[liang8283]

Hi @asheraz048i2c, I looked into this in the cbcopy source and tested it on a test cluster.

Short answer

cbcopy does no pre-check, and it has no skip/log option for constraint violations. In a normal full copy, the duplicate rows are loaded and the primary key is silently not created.

How cbcopy loads data:

• Each table is loaded with a plain COPY ... FROM PROGRAM 'cbcopy_helper ...' [ON SEGMENT] CSV. It has no LOG ERRORS / SEGMENT REJECT LIMIT and no ON CONFLICT. Single-row error handling only covers malformed input anyway, not unique violations.
• Each table is loaded in its own transaction on the target: BEGIN → (TRUNCATE if --truncate) → COPY → COMMIT. Any error rolls back the whole transaction. So a table is all-or-nothing; conflicting rows are never skipped.
• Side note: a PRIMARY KEY can't be NOT VALID (ERROR: PRIMARY KEY constraints cannot be marked NOT VALID).

What happens in a full copy (metadata + data):

During the metadata phase, cbcopy runs DDL and data copy concurrently: each table is queued for data copy as soon as its CREATE TABLE runs. The ALTER TABLE ... ADD CONSTRAINT ... PRIMARY KEY statements run near the end of the pre-data restore. A failed metadata statement is logged as an error.

I reproduced this with a table DISTRIBUTED BY (id) + PRIMARY KEY (id), 2,000,010 rows / 2,000,000 distinct ids:

ERROR: ... ALTER TABLE ONLY public.t_dup ADD CONSTRAINT t_dup_pkey PRIMARY KEY (id);
Error was: ERROR: could not create unique index "t_dup_pkey" (SQLSTATE 23505)
ERROR: Encountered 2 errors during metadata restore; see log file ...
INFO: Database dupsrc: successfully copied 3 tables, skipped 1 tables, failed 0 tables

The result on the target:

• All 2,000,010 rows, duplicates included, were loaded.
• The table had no primary key.
• The table was listed in the succeed file, and the failed file was empty.
• The only signs were the exit code (1) and the metadata error in the log.