Overview
With the Partner Graph File Transfer, your AWS S3 bucket will be filled at regular time intervals with new and updated Match Partner UIDs coming from the cascading process.
The time interval that ID5 pushes files could change over time. Your system should be built to look for new files and process them as they become available and not based on an expected interval from ID5. We recommend you look for new files every 5 minutes.
Streaming Matched Partner Data
The data will be delivered to a directory called /incremental at the root of your bucket with subdirectories broken out by Match Partner (directory names based on the partner’s Global Vendor List ID), then folders per UTC day /YYYYMMDD/, each containing files named with a timestamp throughout the day. For instance:
S3://[bucket]/incremental/[match partner 1 GVL ID]/20221106/000000.csv
S3://[bucket]/incremental/[match partner 1 GVL ID]/20221106/003000.csv
...
S3://[bucket]/incremental/[match partner 1 GVL ID]/20221106/233000.csv
S3://[bucket]/incremental/[match partner 1 GVL ID]/20221107/000000.csv
S3://[bucket]/incremental/[match partner 1 GVL ID]/20221107/003000.csv
...
S3://[bucket]/incremental/[match partner 1 GVL ID]/20221107/180000.csv
S3://[bucket]/incremental/[match partner 2 GVL ID]/20221106/000000.csv
...
S3://[bucket]/incremental/[match partner 2 GVL ID]/20221106/233000.csvEach .csv file contains the incremental mappings between your UIDs and the requested Match Partner’s UIDs in a pipe-separated format. In other words, only new or changed (since the last file) UID pairs will be included in each file.
Available Match Partners are defined by contract between you and ID5. To change the list of Match Partners, please reach out to your ID5 representative.
File Format
| Column | Type | Description |
|---|---|---|
| UID | String | Your User ID for this user |
| MatchPartnerGVLID | Int | The Global Vendor List ID of the Match Partner |
| MatchPartnerUID | String | The Match Partner’s User ID for this user |
Example Output File
Full Matching Table Refreshes
In addition to the incremental updates we push throughout the day, ID5 can stream the full match table on a regular basis. This ensures a couple of things:
- If any data is lost during the streaming process, the full extract will recover the data, rather than waiting for a change from that user to be pushed
- If any users have opted out or had their mappings expire, this will allow you to remove them from your mapping tables since they will no longer be included in the full extract
If you'd like to receive these refreshes, please check with your ID5 representative
Mapping Table Refresh File Location and Format
Both Single-File and Per-Partner Files are supported. The data files will be stored in a different location than the incremental files, allowing you to process them separately for weekly/monthly refreshes. The files will be pushed to:
Per-Partner File Location
s3://[bucket]/full-extract/[datetime]-[clientname]/[match partner GVLID]/[filename].csv
Example:
s3://bucket/full-extract/2025-12-16T14:57:17.765798953-clientname/32/b3a1f108-0e1f-7cee-ad8a-c81a8ba3a68e.csv
Cleaning Up / Deleting Old Files
By default, ID5 does not delete any files we place in the S3 bucket. When we push files to the bucket, we perform a sync operation. This means that if you delete a file from the S3 bucket but it still exists on the ID5 servers, we will re-upload the file to the bucket.
We keep files on our server for approximately 30 days.
If your ETL process includes deleting files from the bucket, please let us know so we can work together on a solution that meets your needs.
We recommend deleting only files older than 30 days to avoid any issues. We can also automatically delete old files from the bucket that we have already removed from our servers if you’d like.