Conquering Governor Limits
A few years ago, when I was still training people on the Salesforce platform, one of the most asked questions would always be whether feature XYZ, which we had just discussed, was limited. My answer would always be, regardless of what the XYZ of the moment was: Yes, on Salesforce everything is limited, it’s a multi-tenant platform.
This is the story of a native Salesforce application that is immune to the most infamous of limits, the Salesforce Governor Limits!
APEX and Limits
As a Salesforce developer dealing with APEX, the most interesting limits are the Governor Limits. They govern everything a developer could possibly want to do. There’s a Governor Limit for everything. Whether it is the maximum number of records one can query or update, the maximum size of blobs, the maximum CPU time a process may use, the maximum size of web service requests or even the maximum time-out allowed on outbound web service calls. Salesforce puts a Governor limit on everything, for good reasons.
As a consequence of these limits, many Salesforce applications that require capacity that could possibly exceed one or more of these limits, end up being built off-platform (e.g. on Heroku or Azure). The application will do the heavy lifting on this external platform and then use the Salesforce web services API to push data into Salesforce.
Imagine building an AppExchange application, on the Salesforce platform, in APEX, that can handle any size data set, for any type of Salesforce object, in any kind of relational structure, containing any number of records.
The records could be Opportunities with OpportunityLineItems, Quotes with QuotelineItems plus random other child- and or parent data, or any set of Custom Objects. For example, 20.000 records, scattered across a data tree resembling the data model pictured in the ERD below, formatted in JSON.
Considering such an APEX application, a few Governor Limits should immediately worry the developer. First of all, the CPU time governor limit is a serious challenge here. Deserializing large JSON texts consumes a ton of CPU time. On top of this, since the data could belong to any SObject, reflection on the Salesforce metadata is required. This is done using describe statements in APEX, which also consume significant CPU time.
Secondly, the limits around DML, both DML rows and DML statements, are at risk of being hit. This is especially true when 100% efficient bulkification is an insurmountable challenge, since the data can take any form, in any constellation of SObjects and database relations.
Besides Governor Limits, other challenges will arise. For example, being able to work on a generic data model, not assuming any particular SObject to exist, the APEX developer will be forced to work with SObjects. SObjects are versatile, but have one major downside: They do not support upsert statements.
Then there’s the processing time required. Obviously, one would like to process the data in as little time as possible. A solution to speed up the processing time, is to process the data in chunks, in parallel. In APEX, this can be achieved with queued jobs. The downside of this approach is a significantly increased chance to hit record locks in Salesforce, considering most of the data will likely belong to some shared parent record.
Challenges like the ones above are what motivates me to get out of bed each morning. This is exactly the reason why I enjoy working at Appsolutely, where my colleagues share this thirst for the extraordinary. We believe the Salesforce platform supports virtually any type of application, even though at first glance going off-platform seems the safest approach.
In partnership with Salesforce ISV e-Con CPQ, Appsolutely has developed an application that tackles all of the challenges outlined above. The application has been published on the AppExchange and can be found here.
Although named “CPQ”, it is basically an integration between a magnificent rules engine (e-Con) and the Salesforce database. It can be used as a CPQ application with infinite options, but can be configured to just as easily serve any other type of functionality inside Salesforce.
The app is virtually immune to the Salesforce Governor Limits, yet does all of the (recursive!) processing of the aforementioned data tree on Salesforce. It is able to process and store 20.000 records in Salesforce in roughly 3 minutes time, without hitting Governor Limits or failing on record locks.
The key to Governor Limit immunity lies in a dozen factors, some of which are outlined below.
Splitting the data set
By splitting up the data set and offering the chunks separately to a custom APEX REST service on Salesforce, the e-Con engine maintains full control over the transaction from a database point of view. This allows for robust error handling and full control over the behavior of the APEX transactions (from a Salesforce execution context point of view).
To speed up processing, the data is chunked and processed asynchronously, using the Queueable interface. Using queued APEX jobs has an additional benefit, some Governor Limits are increased by a factor 6. Since this applies to the CPU time limit as well, using queueables increases the handling capacity of the transaction by 6, allowing more data per chunk.
To avoid record locks, a retrying mechanism has been built into the processor. By retrying DML at configurable random intervals, record locks are avoided and eventually all chunks will successfully process. Anticipating on record locks is quite simple in APEX, as the code sample below shows.
The behavior of the APEX processor and the e-Con rules engine is highly configurable. This allows tailoring of the behavior to fit exactly the requirements of the data set being processed. Although this comes with risks (more specifically: misconfiguration), it puts full control in the hands of the admin configuring the behavior. For every set of data there is an ideal configuration that will allow it to process successfully.
During the processing of the JSON data payload(s), every last millisecond of CPU time has been optimized, using for example caching of metadata information. Think of SObjectType information, SObjectField information and CRUD permissions.
Metadata and SObjects
By making smart use of metadata and SObjects, no mapping or conversions of data are required. The eCon rules engine uses Salesforce’s metadata API to work on any Salesforce instance and on any Salesforce object. In turn, it propagates the metadata information to the custom REST service on Salesforce, when data requires processing.
To serve extremely large and complex data sets, where Salesforce records locks cannot be avoided, the application has a fallback solution. It can process an unlimited number of records successfully, working in serial mode (synchronous).
One drawback of processing synchronously are the reduced Governor Limits, resulting in less data per chunk and a higher total processing time. There are several upsides as well, though. First of all, there is no risk of record locks on Salesforce and secondly the rules engine has full control over the order in which chunks are processed.
Appsolutely is proud to have taken part in developing and publishing this amazing app, that perhaps some of us would have believed impossible not so long ago. When the going gets tough, Appsolutely gets going!