Written by Santosh Sundar,
Jul 11, 2024
Introduction
Not long ago, our team came across a surprising new issue when running bots in live VMs. To provide some context, by default each VM deployed for automation has 2 user accounts that can work concurrently. In this case, we deployed 2 parallel instances of the bot to optimise the throughput.
The bots are programmed to extract data from invoices, run validations and send the finalised data using API calls to the target system. However, due to invalid dates and characters in the API payload the bot kept failing intermittently at the API steps. This development seemed very unusual as the very same bots ran perfectly fine in the UAT environment during the testing phase.
Diving in
When trying to find the root cause, we were able to narrow down the discrepancies between the UAT environment and the live environment to a few identifiable differences. One among them is that we were running 2 parallel sessions in the live VM while in UAT bots were run in single sessions.
We had 4 VMs live running in this case, used by the bots to process the invoices. On each of these VMs, there were 2 robot sessions running in parallel with 2 different service IDs (RPAService3 and RPAService4).
Error message
The robot service IDs were processing multiple invoice documents at the same time. As seen in the error message above, it is clear that the API payload had invalid characters.
When looking at the logs, we discovered the field which was intended to hold the PO number temporarily had the URL of the document.
The only time the robot places this information into API payload is when it retrieves it from the clipboard. As a result, we deduced that clipboard copy and paste actions performed by the bot must be the cause.
Solution development
Luckily, our team quickly identified a solution: to “Disable” the clipboard redirection in the Windows server (which is enabled by default). It should be noted that only system administrators with access to the VM/VDI system settings can change the clipboard redirection. To manage this, reach out to the admins and request that they apply a group policy for all the VMs.
Our team was able to replicate the issue and build this solution, outlined below:
- When the bot using RPAService3 is pasting data that was extracted from the invoices to the Clipboard, it is at the same time that the bot using RPAService4 is retrieving a PO number or document URL from the clipboard.
- Because the clipboard is receiving information from the RPAService3 bot, the RPAService4 Bot fails by using incorrect data in the API call.
- Clipboard redirection:
- This setting allows or prevents the clipboard on the user device from being mapped to the clipboard on the server. By default, clipboard redirection is allowed.
- Follow the path Computer Configuration > Administrative Templates > Windows Components > Remote Desktop Services > Remote Desktop Session Host > Device and Resource Redirection and find Do not allow Clipboard redirection.
- To prevent copy-and-paste data transfer between a session and the local clipboard, select Prohibit. Users can still copy and paste data between applications running in sessions.
- After choosing this setting, configure the maximum allowed bandwidth the clipboard can consume in a client connection. Use the Clipboard redirection bandwidth limit or the Clipboard redirection bandwidth limit percent settings.
- This setting allows or prevents the clipboard on the user device from being mapped to the clipboard on the server. By default, clipboard redirection is allowed.
Post-implementation
Once infrastructure administrators applied the Clipboard Redirection Policy on all live VMs, there was no further cross talk between the RPA bot sessions (RPAService3 and RPAService4).
This essential group policy is recommended to all users who intend to have multiple live VMs with 2 or more user sessions running in parallel at the same time.
