During the first weeks of our training we had to build a dozen of different workflows. Particularly complex workflows and combined data sets took up to couple of minutes to run. Therefore, I had to find a way to make the process run faster. In this post I am going to provide you with some tips to speed up the workflow and potentially save hours of your precious time.
1. Change Runtime Settings and Record limit
- You can set your default Sort/Join Memory Usage lower or higher depending on the use of your computer.
- Action: Workflow Configuration > Runtime tab >Memory Limit > Use Specific Amount.
- To set limits for all input tools go to Workflow Configuration > Runtime tab >Record Limit for all Inputs.
2. Remove Unnecessary Fields with the Select tool
- Use the select tool to remove fields or columns from your data. It can be added later to the workflow if necessary.
3. Use the Sample tool
- Use the Sample tool on large datasets to limit the number of records passing through the workflow.
4. Filter tool
- Use the filter tool to remove unnecessary data. It will allow smaller amounts of data to be passed downstream.
5. Eliminate Browse Tools
- Eliminate the use of Browse tools to reduce the amount of memory used to display the data.
- Action: Workflow > Runtime > Disable All Browse Tools.
6. Disable tool containers
- Tool Containers can be disabled to run only certain portions of the workflow, effectively bypassing tools for a quicker run.
7. Cache Data
- The cache and Run Workflow option will save a temporary copy of the results so all tools from before won't need to reprocess every time you click run.
8 . Auto Field Tool
- You can optimies your workflow for speed by setting the field type to the most efficient type and smallest possible size by using Auto field tool after the Input Data.
9. Enable Performance Profiling
- Performance Profiling shows milliseconds and percentage breakdowns per tool in the workflow. This will allow you to see the slower tools in your workflow and apply methods mentioned above to imrpove its processes.
- Action: Workflow > Runtime tab > Enable Performance Profiling.
Make sure to follow my tips and enjoy the time you have saved.
