Friday, May 12, 2023

Multi selection on forms with the SysOperation framework

In Dynamics 365 Finance and Operations (F&O), the SysOperation framework was designed to allow developers to perform data operations in F&O and is used extensively in the standard application. The SysOperation framework is one of my favorite and frequently used frameworks. The SysOperation framework usually works with a predefined query to fetch data from the system so that the data can be processed in one way or the other.

Since the SysOperation framework was introduced in Dynamics AX 2012, I have used it for all my custom operation logic. Doing this has proven very flexible because each separate operation is dedicated to doing a single thing, and this single operation can then be reused without having the dreaded breaking changes.

In the field, to improve their efficiency, my clients have requested the ability to do multi-selection on frequently used forms containing data. They want to select certain records in the form grid and then process the selected records with the custom functionality on the form that I created. When asked this, I realized that this is not possible with the standard SysOperation framework.

I decided to solve this using a temporary (temp) table. The RecIDs of the selected records are added to the temp table, and then the temp table is joined with the query of the data contract class.

Disclaimer: Thie code below is for educational purposes only. Use at your own risk. Do not use in production. You have no rights toward the author. There are no guarantees on the code.

Okay, let's start.

Temporary table

The code below uses a temp table and you will to create a temp table with a single RefRecId field. The data type of the field must be Int64, you can then extend from RefRecId.

Make sure the Table Type property is TempDB and the CreatedBy property is enabled because the temp table query will filter on the current user.

You could use one of the standard tables that contains a RefRecId field as long as the Table Type of the table is TempDB and the RefRecId field data type is Int64.

With the SysOperation framework, I usually create a minumum of three classes, the data contract, controller and the business logic or service class. There is also a helper class which is used for the multi selection and joining with the query.

Data contract class


[DataContractAttribute]
public final class AVSysOpMultiSelectTemplateContract
{
    private str packedQuery, packedSelectedRecIds;

    public Query getQuery()
    {
        return new Query(SysOperationHelper::base64Decode(packedQuery));
    }

    public void setQuery(Query _query)
    {
        packedQuery = SysOperationHelper::base64Encode(_query.pack());
    }

    [DataMemberAttribute, AifQueryTypeAttribute('_packedQuery', queryStr(CustTable))]
    public str parmQuery(str _packedQuery = packedQuery)
    {
        packedQuery = _packedQuery;
        return packedQuery;
    }

    public Set getSelectedRecIdSet()
    {
        if (!packedSelectedRecIds)
        {
            throw error("You have to set the selected RecIds first.");
        }
        return Set::create(SysOperationHelper::base64Decode(packedSelectedRecIds));
    }

    public void setSelectedRecIdSet(Set _selectedRecIdSet)
    {
        if (_selectedRecIdSet == null)
        {
            throw error("Argument %1 cannot be null.");
        }
        packedSelectedRecIds = SysOperationHelper::base64Encode(_selectedRecIdSet.pack());
    }

    public boolean isMultiSelected()
    {
        return packedSelectedRecIds ? true : false;
    }
}

Controller class with static main method


[SysOperationJournaledParametersAttribute(true)]
public final class AVSysOpMultiSelectTemplateController extends SysOperationServiceController
{
    public static void main(Args _args)
    {
        AVSysOpMultiSelectTemplateController operation = AVSysOpMultiSelectTemplateController::construct();
        operation.parmLoadFromSysLastValue(false); //called from the UI so do not call getLast, we don't want previous selections etc.
        operation.parmShowDialog(true);
        
        AVSysOpMultiSelectTemplateContract contract = operation.getDataContractObject();

        if (AVMultiSelectionHelper::isMultiSelected(_args))
        {
            contract.setSelectedRecIdSet(AVMultiSelectionHelper::getSelectedRecIDSet(_args));
        }
        else
        {
            operation.AVModifyQueryFromArgs(_args);
        }

        if (operation.startOperation() == SysOperationStartResult::Started) //will return result because we are running synchronous.
        {
            Counter cntr;
            [cntr] = operation.operationReturnValue();
            info(strFmt("Processed %1 selected records.", cntr));
            //refresh caller here
        }
    }

    public void new(IdentifierName _className = '', IdentifierName _methodName = '', SysOperationExecutionMode _executionMode = SysOperationExecutionMode::Asynchronous)
    {
        super(_className, _methodName, _executionMode);

        this.parmClassName(classStr(AVSysOpMultiSelectTemplateService));
        this.parmMethodName(methodStr(AVSysOpMultiSelectTemplateService, runQuery));
        this.parmExecutionMode(SysOperationExecutionMode::Synchronous);
        this.parmDialogCaption("Multi select example");
    }

    public static AVSysOpMultiSelectTemplateController construct()
    {
        return new AVSysOpMultiSelectTemplateController();
    }

    public void AVModifyQueryFromArgs(Args _args)
    {
        if (!_args)
        {
            throw error(strFmt("Argument %1 cannot be null.", identifierStr(_args)));
        }

        if (_args.dataset() != tableNum(CustTable))
        {
            throw error(strFmt("Caller buffer must be %1.", tableStr(CustTable)));
        }

        CustTable custTable = _args.record();

        if (custTable.RecId != 0)
        {
            AVSysOpMultiSelectTemplateContract contract = this.getDataContractObject();

            if (contract != null)
            {
                Query query = contract.getQuery();

                if (query != null)
                {
                    QueryBuildDataSource custTable_ds = query.dataSourceTable(tableNum(CustTable));
                    SysQuery::findOrCreateRange(custTable_ds, fieldNum(CustTable, AccountNum)).value(queryValue(custTable.AccountNum));
                    contract.setQuery(query);
                }
            }
        }
    }

    public boolean showQuerySelectButton(str _parameterName)
    {
        boolean ret;

        ret = super(_parameterName);
        
        AVSysOpMultiSelectTemplateContract contract = this.getDataContractObject();

        if (contract.isMultiSelected())
        {
            ret = false;
        }

        return ret;
    }

    public boolean showQueryValues(str _parameterName)
    {
        boolean ret;

        ret = super(_parameterName);
        
        AVSysOpMultiSelectTemplateContract contract = this.getDataContractObject();

        if (contract.isMultiSelected())
        {
            ret = false;
        }

        return ret;
    }
}

Service class containing our business logic


public final class AVSysOpMultiSelectTemplateService
{
    public container runQuery(AVSysOpMultiSelectTemplateContract _data)
    {
        Counter         cntr;
        AVTmpRecId      tmpRecId;    //table containing RecIds of the selected records.
        Set             tmpRecIdSet; //set containing the RecIds of the selected records.
        Query           query = _data.getQuery();

        if (data.isMultiSelected())
        {
            tmpRecIdSet = data.getSelectedRecIdSet(); //get selected RecIds as a Set.
            AVMultiSelectionHelper::populateTmpRecIdFromSet(tmpRecIdSet, tmpRecId); //populate our temp table from the set that contains the RecIds.
            AVMultiSelectionHelper::modifyQueryAddJoinToTmpTable(query); //modify the query to join with our temp table.
        }

        QueryRun queryRun = new QueryRun(query);

        if (_data.isMultiSelected())
        {
            queryRun.setRecord(tmpRecId); //if joined with a temp table, we must set the record on the queryRun object.
        }

        while (queryRun.next())
        {
            CustTable custTable = queryRun.get(tableNum(CustTable));
            info(custTable.AccountNum);
            cntr++;
        }

        return [cntr];
    }
}

Multi selection helper class


public final class AVMultiSelectionHelper
{
    public static Set getSelectedRecIDSet(Args _args)
    {
        Set selectedRecIdSet;

        if (!_args)
        {
            throw error(strFmt("Argument %1 cannot be null.", identifierStr(_args)));
        }

        if (!_args.dataset())
        {
            throw error("Must be called with a caller buffer.");
        }

        if (!_args.record().isFormDataSource())
        {
            throw error("The caller buffer must be form datasource.");
        }

        selectedRecIdSet = new Set(Types::Int64);

        MultiSelectionHelper multiSelectionHelper = MultiSelectionHelper::construct();
        multiSelectionHelper.parmDatasource(_args.record().dataSource());

        Common buffer = multiSelectionHelper.getFirst();

        while (buffer.RecId != 0)
        {
            selectedRecIdSet.add(buffer.RecId);
            buffer = multiSelectionHelper.getNext();
        }

        return selectedRecIdSet;
    }

    public static boolean isMultiSelected(Args _args)
    {
        Set selectedRecIdSet = AVMultiSelectionHelper::getSelectedRecIDSet(_args);
        return selectedRecIdSet.elements() > 1 ? true : false;
    }

    public static void modifyQueryAddJoinToTmpTable(Query _query)
    {
        QueryBuildDataSource    qbds;
        SysDictTable            dictTable;
        QueryBuildDataSource    tmpRecId_ds;

        if (!_query)
        {
            throw error(strFmt("Argument %1 cannot be null.", identifierStr(_query)));
        }

        qbds = _query.dataSourceNo(1); //join on first datasource in query.
        dictTable = SysDictTable::newTableId(qbds.table());
        Debug::assert(dictTable != null);

        tmpRecId_ds = qbds.addDataSource(tableNum(AVTmpRecId), tableStr(AVTmpRecId));
        tmpRecId_ds.addLink(dictTable.fieldName2Id(identifierStr(RecId)), fieldNum(AVTmpRecId, RefRecId));
        qbds.clearRanges(); //joined so clear all existing ranges.

        SysQuery::findOrCreateRange(tmpRecId_ds, fieldNum(AVTmpRecId, CreatedBy)).value(curUserId());                                    
        SysQuery::findOrCreateRange(tmpRecId_ds, fieldNum(AVTmpRecId, CreatedTransactionId)).value(queryValue(appl.curTransactionId()));

    }

    public static void populateTmpRecIdFromSet(Set _recIdSet, AVTmpRecId _tmpRecId)
    {
        if (!_recIdSet)
        {
            throw error(strFmt("Argument %1 cannot be null.", identifierStr(_recIdSet)));
        }

        SetEnumerator enum = _recIdSet.getEnumerator();

        while (enum.moveNext())
        {
            _tmpRecId.RefRecId = enum.current();
            _tmpRecId.insert();
        }
    }

}

Sunday, March 7, 2021

How to download VHD files fast with AzCopy.exe

It's possible to download a complete working Microsoft Dynamics 365 Finance virtual machine (VM) called OneBox from the LifeCycle services (LCS) asset library. By the way, this VM will only run with Hyper-V and will not run with other virtualisation software.

This VM can run locally on your companys infrastructure or even on your laptop provided you have a solid state disk (SSD) with enough space and your laptop has enough memory. My Dynamics 365 Finance VM takes up about 120GB of disk space on my external SSD drive.

I enabled Hyper-V on my laptop in Windows features and I am able to run the Dynamics 365 Finance VM with 24GB of RAM allocated to the VM and 8GB RAM for my Windows 10 host. The VM runs okay but it's definitely not suitable for full time development.

When I downloaded the 12 virtual hardisk (VHD) part files for the first time using with my browser, I discovered that it took a long time to download the files and many of the downloads failed. I had to retry them and it was just painful. Each file is about 3 GB. The separate part files, or volumes, will become one large VHD file when executing the first part file which is an executable.

In the LCS in asset library, on each downloadable file, I saw a button with the text "Generate SAS link". When I clicked on it, I got a message that the "SAS link for the asset has been copied to my clipboard and that I can use AzCopy to download the the asset".

I researched AzCopy a bit and downloaded a copy from the Microsoft site to my laptop and extracted it to a folder.

AzCopy can download files much faster than a browser from BLOB storage would because it uses multi threading to download different parts of the file at the same time. A browser uses a single thread per file being downloaded.

You can compare it to a race to empty two identical swimming pools, containing exactly the same amount of water manually just using buckets. With the first swimming pool, there is just one person emptying the pool but with the other, there are ten persons. Not really fair is it? But the ten persons will empty the pool a lot faster than one right?

I selected the first VHD file and clicked on the "Generate SAS link" button. I pasted the link to a text editor to create a command for AzCopy. I opened a command prompt and downloaded the first file using AxCopy.

To make things easier, I got the idea to add the separate AzCopy command lines in a PowerShell script, one command for each file part. The PowerShell script can download all the part files one after each the other.


Create PowerShell script

  1. Download azcopy and extract it in a folder somewhere like c:\azcopy.
  2. Create a new text file called "downloadvhd.ps1" in the same folder as where azopy.exe is located.
  3. Log in to LCS and go to the Asset Library of your project.
  4. Select the "Downloadable VHD" asset type on the left. If you don't have any part files listed on the right, import them from the Shared Library.
  5. Select the first Part01 VHD file and click on the "General SAS link" button for that file. This will copy the link to the clipboard.
  6. Open the file created in step 2. with a text editor. On the first line, type: .azcopy.exe copy '
  7. After the aposthophe, paste the SAS link from the clipboard by pressing Control + V or Paste from the text editor menu. Make sure there is no space between the aposthrophe and the SAS link.
  8. The first part file is an executable file, the rest of the part files are RAR archive volume files with the .rar extension. After the SAS link, type a closing aposthrophe a space and the destination file name e.g.: ' c:\temp\FinandOps10.0.13.part01.exe
  9. For each part file in LCS, create a line in the script file, their file names will be FinandPos10.0.13.part02.rar, FinandPos10.0.13.part03.rar and so on. Important: Click on the "Generate SAS link" button for each file and paste it in the file for the corresponding part file.
  10. When all part files command are in the script file, save it and execute it.

Example PowerShell file

Replace SAS-link-for-Partxx with your own SAS links and change the file names.
.\azcopy.exe copy 'SAS-link-for-Part01' C:\temp\FinandOps10.0.13.part01.exe
.\azcopy.exe copy 'SAS-link-for-Part02' C:\temp\FinandOps10.0.13.part02.rar
.\azcopy.exe copy 'SAS-link-for-Part03' C:\temp\FinandOps10.0.13.part03.rar
.\azcopy.exe copy 'SAS-link-for-Part04' C:\temp\FinandOps10.0.13.part04.rar
.\azcopy.exe copy 'SAS-link-for-Part05' C:\temp\FinandOps10.0.13.part05.rar
.\azcopy.exe copy 'SAS-link-for-Part06' C:\temp\FinandOps10.0.13.part06.rar
.\azcopy.exe copy 'SAS-link-for-Part07' C:\temp\FinandOps10.0.13.part07.rar
.\azcopy.exe copy 'SAS-link-for-Part08' C:\temp\FinandOps10.0.13.part08.rar
.\azcopy.exe copy 'SAS-link-for-Part09' C:\temp\FinandOps10.0.13.part09.rar
.\azcopy.exe copy 'SAS-link-for-Part10' C:\temp\FinandOps10.0.13.part10.rar
.\azcopy.exe copy 'SAS-link-for-Part11' C:\temp\FinandOps10.0.13.part11.rar

Syntax

azcopy.exe copy [source] [destination]
    
Example:
    
azcopy.exe copy 'SAS-link-for-Part01' C:\temp\FinandOps10.0.13.part01.exe<

Notes

The first part file is an executable file which will extract and combine all the part files into a single VHD file.

You have to check the file names in LCS and adjust the file names in the script accordingly to match the version in LCS. Don't use my example version.

The SAS link is enclosed in single apostrophies. If the file names contain spaces, then the file name part must also be enclosed with single apostrophies.

Download in progress