SysOperation framework execution mode

Personally, I use the SysOperation Framework (SOF) frequently when I am writing business logic code that requires execution in batch.

The execution mode of your operation can be controlled by the framework in different ways depending on the situation. One important factor is SysOperationExecutionMode. When running operations using SOF, you can control the execution mode to best suit your needs.

For example, if you add a customization to a standard F&O process, you don't want that impact that process by introducing a lengthy operation or even worse, an exception. For those situations, I use the ScheduledBatch or the ReliableAsynchronous modes.

When executing an SOF operation with ScheduledBatch or the ReliableAsynchronous modes, the operation gets added to batch and the execution of the caller continues. The caller does not have to wait for the operation to finish. It seems to be asynchronous but it's actually just running in batch.

Below are the different execution modes of the SysOperation framework.

Synchronous

The default mode. Runs on the same thread as the caller. The caller thread is paused until the operation is completed.

ReliableAsynchronous

Runs in the batch framework and the batch job is automatically deleted regardless of the final status of the batch job.

ScheduledBatch

Runs in the batch framework and the batch job remains visible after execution regardless of the final status of the batch job.

Asynchronous

Runs on a separate thread as the caller. The caller thread is not paused.

Code

Your controller class extends the SysOperationServiceController class and the default execution mode is set in the new method of the base class SysOperationServiceController class.

public void new(IdentifierName _className='',
    IdentifierName _methodName='',
    SysOperationExecutionMode _executionMode = SysOperationExecutionMode::Asynchronous)
{
    super(_executionMode);

    className = _className;
    methodName = _methodName;

    progressWait = SysOperationProgressWait::construct();

    this.parmShowProgressForm(false);
}

You can override the execution mode in the new method of your own controller class.

public void new(IdentifierName _className = '',
    IdentifierName _methodName = '',
    SysOperationExecutionMode _executionMode = SysOperationExecutionMode::Asynchronous)
{
    super(_className, _methodName, _executionMode);
    
    this.parmClassName(classStr(AVSysOpMultiSelectTemplateService));
    this.parmMethodName(methodStr(AVSysOpMultiSelectTemplateService, runQuery));
    //ScheduledBatch: the batch job remains visible after execution.
    //ReliableAsynchronous: the batch job is deleted automatically after execution.
    this.parmExecutionMode(SysOperationExecutionMode::Synchronous);
}

You can set the execution mode in code using the parmExecutionMode property.

public static void main(Args _args)
{
    AVSysOpMultiSelectTemplateController operation = AVSysOpMultiSelectTemplateController::construct();

    //can call initializeFromArgs() if menu item contains execution mode info.
    //operation.initializeFromArgs(_args);
    operation.parmShowDialog(true);
    operation.parmExecutionMode(SysOperationExecutionMode::ScheduledBatch);
    operation.startOperation();
}

Menu item

If your operation is launched using using a menu item, you can control the execution mode with the Enum Type and Enum parameter properties on the menu item.

Note:
If you rely on the menu item to initialize your operation, calling initializeFromArgs of the SysOperationServiceController base class is required.

public static void main(Args _args)
{
    AVSysOpMultiSelectTemplateController operation = AVSysOpMultiSelectTemplateController::construct();
    
    //call initializeFromArgs() to retrieve execution mode, class name and class method.
    operation.initializeFromArgs(_args);
    operation.parmShowDialog(true);
    operation.startOperation();
}

Below is the standard SOF initializeFromArgs method of the SysOperationServiceController class. It retrieves the execution mode from the menu item that launched the operation.

public void initializeFromArgs(Args a)
{
    container serviceInfo = SysOperationServiceController::parseServiceInfo(a);

    this.parmClassName(conPeek(serviceInfo, 1));
    this.parmMethodName(conPeek(serviceInfo, 2));
    if (conLen(serviceInfo) == 3)
    {
        this.parmExecutionMode(conPeek(serviceInfo, 3));
    }
    this.parmArgs(a);
}

Strings in X++ when to use " and when to use '

I have seen a lot of code of a lot of developers during the past two decades. There is however one thing that even more seasoned developers keep getting wrong when it comes to using strings in code.

I see mixed uses of "" and '' as demonstrated in the two examples below. Both are programatically correct and are accepted by the compiler and both work fine.

public static str callerBufferIsMandatory()
{
    return "The form has been called incorrectly. A caller buffer is required.";
}

Example 1 of a string with "".

public static str callerBufferIsMandatory()
{
    return 'The form has been called incorrectly. A caller buffer is required.';
}

Example 2 of a string with ''.

Conclusion

Basically, if your string is used in the use interface towards the end user, you use "" for the string. Best practice is to use a label but this is another topic all together and not covered here.

If your string is not used in the user interface and is a basically a constant that won't change and will never be converted to a label, you use '' for the string. Like the example below. The name is a constant and will never change.

public static void writeLog(str _text)
{
    new SysExceptionLog().writeEntry(Exception::Info, _text, 'MyCustomizationName');
}

Generating deep links

When a form is opened in Dynamics 365 Finance and Operations F&O the form displays all records or sometimes applies a filter if a default view is applied. It doesn't drill down to a single record. Sometimes you need a sharable URL link to drill down to a single record. E.g. a customer or vendor or it could be anything really. With deep links, you can drill down to a specific record in a form.

When the system generates a deep link of a specific record, it adds detailed query parameters to the URL. The extra parameters are readable in the unencrypted version above. The datasource name, table field name and filter value are specfied for the form to filter on. If the form menu item has the "Allow root navigation" and "Copy caller query" properties set to Yes, then the form will recognize the extra parameters and apply the filter automatically.

Unencrypted deeplink:

https://usnconeboxax1aos.cloud.onebox.dynamics.com/?cmp=USMF&prt=initial&mi=display:CustTable&q={"Parameters":[{"DataSource":"CustTable","FieldValues":[{"Field":"AccountNum","Value":"US-002"}]}]}

Encrypted deeplink:

https://usnconeboxax1aos.cloud.onebox.dynamics.com/?cmp=usmf&prt=initial&mi=display:CustTable&q=BgAAAKtjn9amNj9Ibe1SID5uUafqfbbsq2Of1qY2P0ht7VIgPm5Rp%2bp9tuymMl72yjSV2bDEqxFBm%2blUyTB3uTX5155lA1AAn%2fzUohRn52fhmJ7y%2fy6kGLVzYTxMde0xqPliZnUU28hblRZ%2b4CxkpeT1576UN2n89XUANdpPxJ4H2mcc0O9WCOdpgtWGEr5nC8hHV%2bvV4u4Pj5DVhdrcWyg7ipSdPfVtNtPO2EmMf2nQQMTSePLlPvKJq8AlHqakLjdeN74lqUCxqroB8l0VgiXfYVvUe9NXs8hwnX6MrP2EAaUdd4dzE7DaNJNuxQnRXbu2BbO0QaUyBERPZ0IofUO3M8G5M9JpePw7nBMvBrCnixIGg4tZMAoPK0cjEAycIvYzG1A4be%2bhTWe1odR6WtnF4B3J8syydPbHUkIqWlnltkJCUToiQ4FiOfhScK%2bzHsyv2%2b8WLpLG2fxrIJGR5DrBusC6PLy%2bWHfZvvHQnUfMZnbqknj6%2b%2fG2qOcE%2fK%2fP%2baAkbfxDeaLTjrgHx2Sbe6KB4eKIi%2fA8bb8VidM29KkXkjXfAWRe%2b6D%2fognEL0kcy1nnYHZKs%2bOEl32ofeMT7REdfmSVLPP7tonH1NgKo7r%2fA9qIR%2b11%2bzbAXGfc0U76ZphIlylscjXMkZGceDK5%2b8mXgHFjF8vtamNaH1uH%2byONKCYsDCq4kI4W%2bfM7XvTulFVWq7hWJb2hnPAsCkdJuWKdgWbgOs4ACSQvH3f63ZQtysryLH0yAsa8TQdmXYJ610xmskt0a6K4hC7ZHFHdfGPYyICwSza7wDH6mZJuWlnkdSKQVixmCV1HRkyjkVTg5fYTXZyfeWGHVpwf%2bBC3%2fp7NyD7wFsvP3im6S1Cdgu5eLIwVKRlZyaSj37SoylD5XwPQmmQXURi54I2f

The encrypted version hides the query information so it's safe to be used externally. The system encrypts the extra query parameters and this results in a unreadable and safe version of the URL.

Notes:

• The menu item used must have the following properties enabled:
• Allow root navigation = Yes
• Copy caller query = Yes
• The UrlUtility::getUrl() function does not work in batch. Do a search on an alternative way to get the system URL in batch.
• Use the SysEntityNavigation data entity to create deep links from external applications.
• The length of deep links URL are more than 1000 characters long and can vary. The length of the above deep link is 1063 characters.
• Only use unencrypted deeplinks for testing as they will result in the error message below.

Error message: The specified record query failed to apply and the can't be opened.

Full code example

public class MyDeepLinkTest
{
    public static str createDeepLinkUrl(MenuItemName _menuItemName, DataSourceName _keyFilterDataSourceName, FieldName _keyFilterFieldName, str _keyFilterFieldValue)
    {
        var generator = new Microsoft.Dynamics.AX.Framework.Utilities.UrlHelper.UrlGenerator();
        var currentHost = new System.Uri(UrlUtility::getUrl());

        generator.HostUrl = currentHost.GetLeftPart(System.UriPartial::Authority);
        generator.Company = curext();
        generator.MenuItemName = _menuItemName;
        generator.MenuItemType = MenuItemType::Display; //avent: display only.
        generator.Partition = getCurrentPartition();
        generator.EncryptRequestQuery = true; //avent: set to false for testing only.

        //repeat for each datasource to filter
        var requestQueryParameterCollection = generator.RequestQueryParameterCollection;
        requestQueryParameterCollection.AddRequestQueryParameter(_keyFilterDataSourceName, _keyFilterFieldName, _keyFilterFieldValue);

        System.Uri fullURI = generator.GenerateFullUrl();
        
        return fullURI.AbsoluteUri; //to get the encoded URI
    }

    public static void main(Args _args)
    {
        str link = MyDeepLinkTest::createDeepLinkUrl(menuItemDisplayStr(CustTable),
            formDataSourceStr(CustTable, CustTable),
            fieldstr(CustTable, AccountNum),
            "US-002");

        Dialog dlg = new Dialog();
        dlg.addField(extendedTypeStr(Notes)).value(link);
        dlg.run();
    }

}

Result of running above test code to create a deep link

Using the deep link

Edit: (untested) alternative method for getting the host URL. Source community.dynamics.com.

IApplicationEnvironment env = EnvironmentFactory::GetApplicationEnvironment();
str currentUrl = env.Infrastructure.HostUrl;
System.Uri currentHostUrl = new System.Uri(currentUrl);
 
UrlGenerator urlGenerator = new UrlGenerator();
urlGenerator.HostUrl = currentHostUrl.GetLeftPart(System.UriPartial::Authority);

Getting stack trace information using xSession::xppCallStack()

From the battle hardened trenches of the F&O development front lines, I present to you a small code nugget that will help you save time and troubleshoot your custom code.

I am actually talking about the standard system method xSession::xppCallStack(). It retieves the current execution call stack. If combined with an error message, it provides useful information for troubleshooting custom code running in production.

The code example below, is just an example of using the standard system method xSession::xppCallStack() to retrieve the current execution call stack of the code being executed. Having the execution call stack information saves a lot of time because you can see where error message originates from.

public static void throwError(str _error)
{
	container callStack = xSession::xppCallStack();
	conDel(callStack, conLen(callStack), 1); //remove last one.
	throw error(_error + '\n' + con2Str(callStack, '\n'));
}

How to effectively detect if data has changed using RecVersion

Sometimes there is a requirement to effectivle detect if data has changed in a table. The first thing that might come to mind is, why not just use the "ModifiedDateTime" field of the data? Well, that's perfectly fine if don't have a lot of data in your table.

But, what if there is a lot of data in our table? Like millions of records? Filtering on an unindexed field like "ModifiedDateTime" and with a lot of data, will force the database server to scan all the records in the table. The database server has to do this to determine which records are modified after the specified date and time. Table scans on large tables is not effective and painfully slow because it's not using any of the table indexes.

RecVersion

So, what's the solution? How do we effectively detect which rows have changed in our data? By using the system field RecVersion. Every table in Dynamics 365 Finance and Operations (F&O) has a field called RecVersion.

Every time a record is changed in a table, the RecVersion of the record is updated by the system to a new value, or version. The RecVersion field contains the record version of the record.

But, there is a small problem, how do know what the previous version of RecVersion was before the data was changed? Well, we don't. But luckily there is a simple way so solve this. By creating a custom table with the RecId and RecVersion of our data.

Steps

• Create a custom table with a reference RecId and RecVersion field to our data. Field names: RefRecId and RefRecVersion
• In code, create a method to create a "snapshot" of the RecId and RecVersion values of the data. Use insert_recordset to copy the data.
• Query your table with a join to the custom table. Join on YourTable.RecId and YourTable.RecVersion and SnapShotTable.RefRecId and SnapShotTable.RefRecVersion

To get all the data from the target table that has changed, join the two tables on RecId and RecVersion. The records where RecVersion is different, has changed.

Base64 encoding and decoding

What is Base64 encoding and decoding? On a very high level: it is a method of converting binary data to ASCII text and the ASCII text back again to binary. There are many articles online on how this works technically but this article will explain the basics and how Base64 encoding and decoding is used in Dynamics 365 Finance and Operations (F&O).

For example, if you attach a spreadsheet or other binary file like a PDF document to an email. The binary file is converted to ASCII text using Base64 encoding and is embedded in the message body of the email as text. When the email message is received by the recipient, the email application used to view the email message, detects that there is a file attached embedded as text and converts the text back to binary so that the attachment can be downloaded as a file. Most email applications support previewing attachments, but will have to convert the attachments from text to binary on the fly before being able to display the contents of the file.

So, how does Base64 encoding and decoding relate to F&O? It is frequently used in the standard application and also by developers when using the SysOperation framework just to mention just a few examples.

Code example

In a SysOperation contract class, the SysOperation framework allows a Query object to be used to query data for the operation. Because the related operation can also be run in "batch", in other words in "unattended" mode without an user interface, the Query object of the class needs to be saved to the database so that the operation can be executed by the batch framework and use the exact same query object that was specified by the user when adding the operation to batch. In other words, the system saves all query filters, joins and other changes to the query and will then use your exact query when executing in batch.

An instance of an class object cannot be saved to the database. So, how does the system save the object to the database? By using Base64 encoding and decoding. The Query object of the contract class is serialized to a binary object using the "pack" method of the Query class. This method returns a binary representation of the query in a container type. All class variables of the Query object are serialized and converted to a binary object (container). The resulting binary object of the Query object is then Base64 encoded to text. This text can be saved to the database as a regular text field.

When the Query object instance is required by the SysOperation framework, the process is reversed. The text is Base64 decoded to binary, the original Query object with all the specified filters and other changes can then be instantiated by using the binary data to construct a Query object.

1. Class variable for the Base64 encoded text.

2. Code that converts the text back to binary object.

3. Code that converts the binary object to text.

Using Boolean OR to make where clause fields optional

Sometimes you have a custom "while select" statement and you need to have one or more of the where clause fields in the statement to be optional. In this example, I will use a simple method to display the customer account and group for customers.

The method below displays the customer accounts for customers and uses the _custGroup argument to filter customers that match the customer group. The method works as expected but there is one drawback: it can only show customers for the specified customer group.

So, what if we want to show info for all customers with this method? Can't we just call the method and pass an empty string? The answer is yes, that's possible but will not work as one would expect. The problem with this is, that if you call the method and pass an empty string for the customer group, the while select statement will return all customers where the customer group is empty. Since the customer group field is mandatory, and all customers have a customer group, the query will return nothing.

Show customer info method

//show customer info with mandatory argument
public static void showCustomerInfo(CustGroupId _custGroup)
{
    CustTable custTable;

    while select custTable
        where custTable.CustGroup == _custGroup
    {
        info(strFmt("%1 - %2", custTable.AccountNum, custTable.CustGroup));
    }

}

How do we solve this problem? Your first thought might be to simply duplicate the while select statement in the method for each argument used in the where clause. In this example two statements, one statement with, and another statement whithout the customer group where clause. And simply add an "if" statement to check if the customer group argument has a value or not. Yes, this will work but this is definitely not the way to go. The reason is because if you have a large while select statement with many lines and a method with multiple arguments. The code will be hard to read and error prone when modifications are done to it.

Optional argument

So, what is the solution? By using the Boolean OR operator in the while select statement. The where clause for customer group is extended with an Boolean OR to contain two parts. The first part of the Boolean OR evaluates the method customer group argument variable, if this part of the Boolean OR evaluates to TRUE, the second part of the OR is simply ignored, effectively ignoring the where clause part for customer group in the second part. However, if the first part of the Boolean OR statement evaluates to FALSE, the second part of the boolean OR is then evaluated which effectively then applies the where clause of the customer group to the statement. Making the where clause optional and having compact and readible code.

//show customer info with optional argument
public static void showCustomerInfo(CustGroupId _custGroupId = '')
{
	CustTable custTable;

	while select custTable
		where (_custGroupId == '' || custTable.CustGroup == _custGroupId)
	{
		info(strFmt("%1 - %2", custTable.AccountNum, custTable.CustGroup));
	}

}

Complete example

You can run the class using the SysClassRunner class.
e.g. https://[your-environment].operations.eu.dynamics.com/?mi=SysClassRunner&cls=MyTestJob

public final class MyTestJob
{
    public static void main(Args _args)
    {
        CustGroupId custGroup;
        Dialog dlg = new Dialog('Enter customer group');
        DialogField fldCustGroup = dlg.addField(extendedTypeStr(CustGroupId));

        if (dlg.run())
        {
            custGroup = fldCustGroup.value();
            MyTestJob::showCustomerInfo(custGroup);
        }
    }

    //show customer info with optional argument
    public static void showCustomerInfo(CustGroupId _custGroup = '')
    {
        CustTable custTable;

        while select custTable
            where (_custGroup == '' || custTable.CustGroup == _custGroup)
        {
            info(strFmt("%1 - %2", custTable.AccountNum, custTable.CustGroup));
        }
    }

}