How to improve performance when processing large numbers of files
To improve the performance of jobs that routinely process large numbers of files, you should reduce any unnecessary steps in the job.
- In the Archiving panel on the Transaction screen, check Skip Primary Archiving so that one or more copies of each file are not written to the primary archive location.
- In the Archiving panel on the Transaction screen, do not specify an Additional Archive Location so one or more copies of each file are not written to an additional archive location.
- In the Source and Destination Partner Profile panels on the Transaction screen, select None in the File Integrity Checking drop-down, which eliminates several FTP commands for each file being transferred.
- In the Email Notifications panel of the Transaction screen, select Use Abbreviated Notifications, which will limit the length of any business user or IT support email notifications.
- If you expect files in the source directory that are already at the destination, use one of the overwrite conditions in the File Information panel of the Transaction screen (e.g., overwrite if source newer) to reduce the number of files to be processed.
- If you are using SFTP, be sure that Accelerate Transfers is checked in the source or destination partner profile panels of the Transaction screen to reduce the time to transfer files.
- Be sure that Turn on Advanced Troubleshooting at the bottom of the Transaction screen is unchecked, so no unnecessary debug files are created.