Have you ever received an API usage notification email indicating that you've surpassed 75 percent of the available API call threshold? Or perhaps you've faced a situation where the API limits were entirely exceeded, causing the API to cease functioning, and you were unsure about how to proceed with an investigation?

Mulesoft logs didn’t help

Something similar happened on my previous project. I promptly contacted our Mulesoft team, who were responsible for integrations, and assigned them the task of examining the logs and dashboards to identify the source of this additional load.

After several hours of investigation, I hadn't received any conclusive findings. The response was, "There is no unusual traffic. We don't observe any extra requests being generated by Mulesoft."

While the absence of abnormal activities tends to be a good thing, in that case problem was still there, but very well hidden.

Event Monitoring

Luckily, Salesforce offers a built-in summary of API calls through its Event Monitoring tool. While the documentation implies that you need to acquire the Event Monitoring add-on for tool access, the API Total Usage log that we require is available free of charge. The only prerequisite is to activate the "Generate event log files" option within the Event Monitoring section.

An important note: it's imperative to enable this setting in advance, as the logs cannot be generated retroactively. My recommendation is to activate it even before commencing development.

Consequently, all API calls directed towards Salesforce within the preceding day (from midnight to midnight) were recorded in a log file named "APITotalUsage". This log is accessible through the following Heroku App: https://salesforce-elf.herokuapp.com/ (additional methods for obtaining the log are elaborated upon in this Trailhead module: https://trailhead.salesforce.com/content/learn/modules/event_monitoring/event_monitoring_download).

Working with API Total Usage file

There are several options available to visualize the data, including CRM Analytics (if you possess the license). However, if you have at least basic knowledge of APIs in your environment, simple Excel will do. Particularly when you’re in distress and have no time to experiment with different solutions.

Inserting CSV into the Excel is very straightforward. Open the Data tab, select From Text/CSV and then click on Load located at the bottom of the modal window.

Let’s examine the columns, that will be important to us.

Request Id – We will use it for counting unique requests

Timestamp Derived – Doesn’t look important, but Excel can filter or group the data by actual time.

User Id – Who's to say that the Mulesoft integration user is responsible for the excessive requests?

API Resource – Is the action an instance of create, delete, query, update, upsert, or something else?

Entity Name – The object involved in the operation can aid in identifying the problematic API.

Counts Against API Limits – We generally want to filter out requests not counting against API limits

Pivot table

The easiest way to visualize the data is by using a Pivot table. Select 'Insert' -> 'Pivot table' -> 'From Table / Range'. Select all the loaded data and place the Pivot table into a new sheet.

To begin designing the Pivot table, click within the table and drag the fields into one of the four sections: Filters, Columns, Rows, and Values.

I typically start by placing 'Request Id' into the Values section to count the number of requests.

To filter out requests that do not count against API limits, I drag 'Counts Against API Limits' into the Filters section. I also include 'User Id' and 'Timestamp Derived' there, as I occasionally want to check the numbers per user and hour.

Finally, I add 'Entity Name' into the Rows.

Observe

From here, I am attempting to identify which entities are related to the excessive requests. Although it's not rocket science, you can't simply point to the highest number. The ideal scenario is when you can compare reports from multiple days because it's usually normal for some objects, like 'Account', to have significantly more requests than others.

Once you identify a suspicious entity, you should add 'API Resource' to the Rows to understand which request type is causing the issue. You can also filter the data by 'Timestamp Derived' to determine when the problem started. Knowing the recent updates to the interfaces, you should be able to figure out the rest.

Connector Issues

To be honest, whenever I've encountered issues with excessive requests, they've always been caused by the Mulesoft Connector. The first time, I noticed an extremely high number of requests that weren't related to a specific entity. When I added 'API Resource' to the Rows, I saw that something was spamming 'getTimestamp' requests. 'Timestamp Derived' pointed to the last deployment.

With this information, our Mulesoft team realized that they had mistakenly deployed a very old version of the Mulesoft Connector, which was making 'getTimestamp' requests every second. Updating the Mulesoft Connector to the most recent version resolved the issue.

The second time, I identified the issue with an object – let's call it 'Garden'. We had too many 'getSobjectDescribe' calls since the last deployment. Once again, the issue was caused by deploying an incorrect version of the Salesforce Connector.

However, in both cases, without a proper Event Log analysis, I would have been lost.

Looking for an experienced Salesforce Architect?

• Are you considering Salesforce for your company but unsure of where to begin?
• Planning a Salesforce implementation and in need of seasoned expertise?
• Already using Salesforce but not fully satisfied with the outcome?
• Facing roadblocks in your Salesforce implementation and require assistance to progress?

Feel free to review my profile and reach out for tailored support to meet your needs!