Data Loading Strategies: Essential, Sample, and Production Data Without Conflicts

Every SAP Commerce project juggles at least three kinds of data, and the platform's defaults pretend there are only two. ant initialize and ant updatesystem import coredata and sampledata; nothing out of the box says "this is what production gets on go-live day" or "this is what release 14 adds". Projects that never fix this end up with sample products in production, releases that cannot be replayed, and a staging environment whose data resembles nothing real. This guide sets up a loading system that scales from the first local init to the fiftieth production release.

The Three Kinds of Data#

Two principles behind the table. Sample data exists to make a fresh environment usable for development and automated smoke tests within minutes, so it must be light and must not depend on production data. Production data is volatile and multi-sourced, so it can never serve as a test baseline; what you test on STAGE is your release's data changes against a fresh copy of production data, not against a hand-groomed fiction.

A governance rule that saves pain later: before go-live, business-team content contributions are development tasks that land in versioned ImpEx, not direct edits to a shared environment. The solution is still changing shape underneath them; direct contributions get lost or, worse, silently depended upon. After go-live the business team works in PROD through Backoffice and SmartEdit, and the development team's sample data stops pretending to track their output exactly.

Convention Over Configuration: The Folder System#

The platform's AutoImpEx mechanism (ImpExSystemSetup) imports ImpEx files matching per-extension patterns. The convention that carries all four use cases adds two folders to the standard two:

acmecore/resources/acmecore/import/
├── coredata/       # essential data: always imported
├── sampledata/     # local + DEV
├── initdata/       # STAGE/PROD initial load (go-live)
└── updatedata/
    ├── R14/        # release-numbered update data
    └── R15/

Within the folders, remember that ImpExSystemSetup sorts files alphanumerically (java.util.Arrays.sort), so ordering is encoded in file names: 001-catalog.impex, 002-products.impex, 010-relations.impex. Leave numeric gaps; you will insert files later.

Ordering across extensions is controlled by extension dependencies. If extension B's data must load before extension A's, declare it in A's extensioninfo.xml:

<extension name="extensionA">
  <!-- forces B's ImpEx to load before A's -->
  <requires-extension name="extensionB"/>
</extension>

One refactoring task to schedule at project start: the @SystemSetup classes that modulegen generates import sample data unconditionally. Strip those explicit imports so AutoImpEx patterns are the single mechanism deciding what loads where. CCv2 also still supports the legacy JSON-driven configuration for update parameters, but running two configuration systems for the same concern is exactly how environments drift; treat the JSON route as an escape hatch, not an architecture.

Wiring Environments to Folders#

The selection happens through two properties. Locally and on DEV:

# local.properties (local + DEV): sample data only
update.executeProjectData.extensionName.list=acmecore
acmecore.projectdata-impex-pattern=acmecore/import/sampledata/**/*.impex

On STAGE and PROD for the initial go-live load:

acmecore.projectdata-impex-pattern=acmecore/import/initdata/**/*.impex

And for each subsequent release, pointed at that release's folder:

update.executeProjectData.extensionName.list=acmecore
acmecore.projectdata-impex-pattern=acmecore/import/updatedata/R15/**/*.impex

These properties belong in the per-environment property files referenced from manifest.json (useConfig.properties with personas), so DEV builds carry the sampledata pattern and production builds carry the release pattern, from the same repository state.

The CCv2 deployment modes map cleanly onto the system:

Verification is cheap: the console log names exactly what AutoImpEx picked up, and reading it should be part of every deployment check:

INFO [ImpExSystemSetup] AutoImpEx for extension 'acmecore' will use userdefined filepattern
  'resources/acmecore/import/updatedata/R15/**/*.impex' for creating the project data...
INFO [ImpExSystemSetup] importing resource : /acmecore/import/updatedata/R15/001-new-delivery-modes.impex

The Release Rhythm#

First load (go-live). Everything the live site needs on day one sits in initdata/. STAGE gets it first with mode Initialize database, and the full go-live rehearsal runs against it. PROD initialization happens exactly once; after go-live, the initialize option should be treated as radioactive.

Each release. New folder updatedata/RXX/, environment properties pointed at it, mode Migrate data. The non-negotiable gate: RXX data is tested on STAGE against current production data (see the next section for getting it there) before PROD sees it. Data changes that worked against last month's data and fail against today's are a classic release-night failure.

After the release. Two pieces of hygiene. First, fold anything durable from the release's data into sampledata so fresh developer environments keep exercising the new features; sample data is not a copy of production, but it must stay representative of critical flows. Second, remove the previous release's updatedata/ folder from the repository. Executed release data is history, and history that can accidentally re-run is a liability. Git remembers it if an audit asks.

For organizations running more complex release trains (parallel releases, hotfix data), the platform's Patches framework gives data changes a versioned, tracked, once-only execution model with rollback bookkeeping. It costs more structure than folders and properties; adopt it when the folder system starts feeling stretched, not before.

Keeping STAGE Data Real#

Post-go-live, STAGE exists to answer one question: will this release work against production data? It can only answer it if the data is fresh. The Cloud Portal snapshot and restore capability is the mechanism: snapshot PROD, restore to STAGE on a regular cadence (monthly is a common default, always before major releases).

Restored production data contains real customers, so anonymization is not optional; it is a data-protection obligation. The restore flow supports running your own anonymization, and you should own that job, because only you know your custom types. The shape of a working anonymizer, trimmed to its logic:

public class AnonymizerJob extends AbstractJobPerformable<CronJobModel> {

    @Override
    public PerformResult perform(final CronJobModel cronJob) {
        for (final CustomerModel customer : customerService.getAllCustomers()) {
            // customer.uid often contains the email address, so replace it too
            final String replacement = "REPLACED_" + customer.getPk();
            customer.setUid(replacement);
            customer.setOriginalUid(replacement);
            customer.setCustomerID(replacement);
            customer.setName(replacement);
            modelService.save(customer);

            customer.getAddresses().forEach(this::anonymizeAddress);
            for (final OrderModel order : customer.getOrders()) {
                anonymizeAddress(order.getPaymentAddress());
                anonymizeAddress(order.getDeliveryAddress());
                anonymizePaymentInfo(order.getPaymentInfo());
                anonymizeFraudReports(order.getFraudReports());
            }
        }
        return new PerformResult(CronJobResult.SUCCESS, CronJobStatus.FINISHED);
    }

    private void anonymizeAddress(final AddressModel address) {
        if (address == null) return;
        final String replacement = "REPLACED_" + address.getPk();
        address.setFirstname(replacement);
        address.setLastname(replacement);
        address.setLine1(replacement);
        address.setLine2(replacement);
        address.setTown(replacement);
        address.setPostalcode(replacement);
        address.setEmail(replacement);
        address.setPhone1(replacement);
        address.setCellphone(replacement);
        // ...every populated PII field, including company, district, pobox
        modelService.save(address);
    }
    // payment info: billing address + comments; fraud reports: explanation + comments
}

The checklist for your version: customer identity fields (uid doubles as email), every address field, payment info and its billing address, order comments, fraud report texts, ticket contents, and every custom type your project added that carries personal data. The last item is why a copied anonymizer is never enough; audit your items.xml for PII and extend the job accordingly. For a deeper treatment, see the data protection guide in this series.

Migrating Data from Legacy Systems#

Initial data usually comes from somewhere: a legacy shop, a PIM, an ERP. (SAP Commerce on-premise to CCv2 migrations have their own standard procedure and tooling; this section is about non-Commerce sources.) Three viable paths:

The default answer is ImpEx via the repository and the folder system above: simplest, fully versioned, and it lets you redesign the data model rather than importing two decades of legacy decisions. Choose hot folders or the Integration API only when the channel will live on as a real integration after the migration, because both create an integration point you then own forever.

Whatever the channel, the quality gate comes first: migrate only the data the business case names, reformatted to the new model, with the garbage left behind. Middleware earns its keep in the reformatting step, not the transport.

CMS data is the exception to everything. Legacy CMS structures map so poorly onto WCMS pages, slots, and components that automated migration reliably costs more than rebuilding. Rebuild the content in SAP Commerce, and where the volume truly forbids that, port page by page with human judgment.

One related trick for the maintenance phase: when webmasters have evolved production content and your sampledata has fallen behind, export the relevant pages via an ImpEx export script in HAC and hand-prune the result into sampledata:

INSERT_UPDATE ContentPage; catalogVersion(version, catalog(id))[unique=true]; uid[unique=true]; name; masterTemplate(uid); label; defaultPage; approvalStatus(code); homepage
"#% impex.exportItems( ""ContentPage"", false );"

Failure Modes This System Prevents#

• Sample products visible in production, because DEV and PROD imported the same folders
• A release that cannot be replayed onto a fresh environment, because its data changes lived in someone's HAC session
• Release data that passes STAGE and fails PROD, because STAGE data was six months stale
• The same "update" ImpEx running twice with side effects, because executed release data stayed on the active path
• A GDPR finding, because restored production data sat in STAGE un-anonymized

None of the machinery here is exotic: folders, properties, naming discipline, and two Cloud Portal features used on schedule. The value is that data movement stops being tribal knowledge and becomes something the repository itself documents.