ImpEx: The Working Reference for People Who Load Data for a Living

ImpEx is the closest thing SAP Commerce has to a lingua franca: initialization data, migrations, hot folder feeds, support fixes, and half of every go-live run through it. It is also a language most developers learn by copying the nearest existing script, which is how estates end up with imports that work by coincidence. This is the reference for writing scripts on purpose: the semantics that matter, the machinery underneath (passes, dumps, distribution), and the recipes that hold up under production volume. The strategy layer (when to use ImpEx versus Integration APIs versus hot folders) lives in the data loading guide; this is the language itself.

Headers: Modes and the Modifiers That Earn Their Keep#

INSERT_UPDATE Product; code[unique=true]; name[lang=en]; unit(code)[default=pieces]; catalogVersion(catalog(id),version)[unique=true]

Four modes. INSERT creates and fails on duplicates; UPDATE modifies and skips (or errors on) missing rows; INSERT_UPDATE upserts on the attributes marked unique; REMOVE deletes matches. Production imports are overwhelmingly INSERT_UPDATE with a deliberate unique key, because idempotency (the cronjob guide's design test) starts here: a rerunnable script is one whose unique keys identify the same logical rows every run.

Modifiers that do real work:

• unique=true: composes the resolution key. Every INSERT_UPDATE header should make its key set obvious and minimal; a missing unique on catalogVersion is the classic duplicate-product generator.
• lang=en / lang=$lang: targets one localization of a localized attribute; one column per language beats one script per language.
• default=...: value when the cell is empty; forceWrite=true pushes through read-only attribute checks (jalo mode only, use with intent); allownull=true where the model permits.
• mode=append / mode=remove on collection and relation columns: incremental membership edits instead of wholesale replacement. Without it, importing categories(code) replaces the set, which is either exactly what you want (declarative feeds) or a catastrophe (partner enrichment wiping merchandising), so decide per column, on purpose.
• collection-delimiter, path-delimiter, key2value-delimiter: when your data contains the defaults (, : ->), change the delimiter instead of escaping your way to madness.
• dateformat=dd.MM.yyyy HH:mm:ss, numberformat: locale explicitness; imports that rely on server default locale break the day the node locale changes. Pair with a session language line (#% impex.setLocale(Locale.ENGLISH);) for full determinism.
• translator=...: custom value-to-model conversion; the standard library covers most needs (media below), and a small custom AbstractValueTranslator beats preprocessing files in surprising ways exactly once per project.
• batchmode=true on the header: lets UPDATE/REMOVE affect all rows matching non-unique criteria (bulk price deactivation, sweeping flags). Powerful, and the reason review policies exist.

Item references resolve through parentheses: unit(code), catalogVersion(catalog(id),version), arbitrarily nested. Every reference is a lookup at import time; deep nesting on million-row files is a performance decision you are making implicitly (see performance below).

Macros, Config Values, and Reusable Scripts#

$catalogVersion = catalogVersion(catalog(id[default=myCatalog]), version[default=Staged])[unique=true]
$approved = approvalstatus(code)[default='approved']
$siteUid = $config-website.site.uid

INSERT_UPDATE Product; code[unique=true]; $catalogVersion; $approved

Macros ($name = ...) are textual substitution for header fragments and values; they are what keeps a 40-header initialization file maintainable and consistent. $config-someProperty reads platform properties at import time, which is the sanctioned way to make one script environment-aware (site UIDs, URLs, feature flags) instead of maintaining per-environment copies that drift.

Document IDs solve the forward-reference problem inside one script: assign &ref handles and use them before the referenced item has any external key.

INSERT_UPDATE Address; &addrRef; owner(Customer.uid); streetname
                     ; addr1   ; hans@example.com   ; Hauptstrasse 1

UPDATE Customer; uid[unique=true]; defaultPaymentAddress(&addrRef)
               ; hans@example.com; addr1

Media, User Rights, and Other Special Blocks#

Media content imports through the media translator, from files shipped alongside the script (classic :jar: or file references) or remote URLs:

INSERT_UPDATE Media; code[unique=true]; @media[translator=de.hybris.platform.impex.jalo.media.MediaDataTranslator]; mime[default='image/jpeg']; $catalogVersion
                   ; product-front   ; jar:/impex/images/product-front.jpg

On CCv2, bulk media rides the hot folder ZIP conventions or URL media (the cloud hot folders guide covers both); the translator mechanics are the same.

User rights have their own block syntax, and it is the correct home for permission definitions (instead of clicking them into Backoffice and losing them at the next initialization):

$START_USERRIGHTS
Type      ; UID          ; MemberOfGroups ; Target        ; read ; change ; create ; remove
UserGroup ; supportgroup ; employeegroup  ;
          ;              ;                ; Order         ; +    ; +      ; .      ; .
          ;              ;                ; Order.status  ; +    ; +      ;        ;
$END_USERRIGHTS

Types, then type-level and attribute-level grants (+ allow, - deny, . inherit). Version-control these blocks with the same seriousness as code; they are your authorization model (the user management guide picks this up).

Scripting and Conditional Logic#

Lines starting with #% execute code (Groovy via the marker, BeanShell historically) with the impex API in scope; if: / endif: gates blocks:

#% impex.enableCodeExecution(true);
#% if: de.hybris.platform.util.Config.getBoolean("myfeature.enabled", false)
INSERT_UPDATE SomeFeatureConfig; code[unique=true]; active
                               ; default          ; true
#% endif:

Judgment call earned from cleanup duty: scripting in ImpEx is for conditioning (environment gates, small computed values), not for programs. A script whose logic outweighs its data wanted to be a ServiceLayer job or a translator; future readers get one mental model per file.

The Machinery: Passes, Dumps, and Modes#

Understanding the importer explains its error behavior:

• Multi-pass resolution: value lines whose references cannot resolve yet are set aside and retried in later passes (an order line before its order, a superclass category after its children). Only lines still unresolved when passes stop are failures. Consequence: line order rarely matters for correctness, but it does for speed; feeding data in dependency order saves passes.
• The dump file collects unresolved and invalid lines with reasons appended. Operationally this is your contract: import, inspect dump, fix, re-import the dump file itself. Zero-dump is the success criterion for automated feeds, and hot folder monitoring should treat a non-empty dump as a failure signal even when the import "finished".
• Legacy versus ServiceLayer mode: legacy mode writes through the old jalo layer, bypassing interceptors and validators; SL mode fires them. The default is SL, correct for business feeds (your interceptors encode invariants). Legacy mode (impex.legacy.mode or per-import flag) is a deliberate escape hatch for migrations where interceptor side effects would be wrong or lethal at volume; using it casually means loading data your own code considers invalid.
• ServiceLayer Direct (SLD) persistence skips the model layer for straight-line writes and shows up as the "direct persistence" toggle in import UIs; a meaningful throughput lever for huge, simple loads, with the same caveat as legacy mode: know which side effects you are opting out of.
• Distributed ImpEx splits a file into work groups executed across the cluster (the clustering guide's answer to "one node imports while five watch"). It shines on big, independent value lines (products, prices, stock) and is pointless for reference-dense graphs that serialize on resolution anyway.

Performance Recipes#

For the ten-million-row price load: INSERT_UPDATE with minimal unique keys on indexed attributes, references kept shallow (pre-resolve catalog version into a macro, not per-line nested lookups), file split by independence and fed to distributed ImpEx, SLD on, database-logging off for the run, retention job ready for the import cronjob logs (the cronjob guide's JobLog warning applies doubly to imports). Measure a 100k-row slice first; per-line cost times volume is your window, and the high-traffic guide's freeze calendar tells you when that window is allowed to exist.

For the surgical support fix: smallest possible script, explicit catalog versions, INSERT_UPDATE never bare INSERT, validated in hAC's validate-only mode, exported counterpart of the touched rows saved before (the export console is your undo button), and the script attached to the ticket. Data surgery without an audit trail is how the next incident starts.

ImpEx has outlived every replacement announced for it because it does one thing with total clarity: declarative data against the type system, in a format a human can read in a code review. Write it like the code it is (versioned, reviewed, idempotent, environment-aware via config macros) and it will still be the most dependable tool in the box when everything around it has been renamed twice.