Friday 12 May 2023

Microsoft Dynamics 365 Finance - Multi selection on forms with the SysOperation framework

Background

The SysOperation framework works with a query to fetch any data that will be processed.

In some cases, it is required to only process the selected (marked) records on a form but this is not possible without some extra coding.

I have solved this using a temporary (temp) table. The RecIDs  of the selected records are added to the temp table and then the temp table is joined with the query of the data contract class.


License/disclaimer: Free to use. Use at own risk. No rights or guarantees.


Let's start.


Temporary  table

The code below uses a temp table and you will to create a temp table with a single RefRecId field. The data type of the field must be Int64, you can then extend from RefRecId.


Make sure the Table Type property is TempDB and CreatedBy property is enabled because the temp table query will filter on the current user.

Note: You could use one of the standard tables that contains a RefRecId field as long as the Table Type of the table is TempDB and the RefRecId field data type is Int64.


With the SysOperation framework, I usually create a minumum of three classes, the data contract, controller and the business logic or service class. There is also a helper class which is used for the multi selection and joining with the query.


Data contract

[DataContractAttribute, SysOperationAlwaysInitializeAttribute]

public final class AVSysOpMultiSelectTemplateContract implements SysOperationInitializable
{
    private str packedQuery, packedSelectedRecIds;

    public void initialize() //always called because of SysOperationAlwaysInitializeAttribute
    {
    }

    public Query getQuery()
    {
        return new Query(SysOperationHelper::base64Decode(packedQuery));
    }

    public void setQuery(Query _query)
    {
        packedQuery = SysOperationHelper::base64Encode(_query.pack());
    }

    //the query specified here is the query shown in the SysOp dialog and also the
    //query used in the SysOp service class.
    [DataMemberAttribute, AifQueryTypeAttribute('_packedQuery', queryStr(AVCustTable))]
    public str parmQuery(str _packedQuery = packedQuery)
    {
        packedQuery = _packedQuery;
        return packedQuery;
    }

    public Set getSelectedRecIdSet()
    {
        if (!packedSelectedRecIds)
        {
            throw error("You have to set the selected RecIds first.");
        }
        return Set::create(SysOperationHelper::base64Decode(packedSelectedRecIds));
    }

    public void setSelectedRecIdSet(Set _selectedRecIdSet)
    {
        if (_selectedRecIdSet == null)
        {
            throw error("Argument %1 cannot be null.");
        }
        packedSelectedRecIds = SysOperationHelper::base64Encode(_selectedRecIdSet.pack());
    }

    public boolean isMultiSelected()
    {
        return packedSelectedRecIds ? true : false;
    }
}


Controller

[SysOperationJournaledParametersAttribute(true)]

public final class AVSysOpMultiSelectTemplateController extends SysOperationServiceController
{
    public static void main(Args _args)
    {
        AVSysOpMultiSelectTemplateController operation = AVSysOpMultiSelectTemplateController::construct();
        operation.parmLoadFromSysLastValue(false); //called from the UI so do not call getLast, we don't want previous selections etc.
        //operation.initializeFromArgs(_args); //can call initializeFromArgs() if menu item contains SysOp execution info. if not, it removes SysOp execution info.
        operation.parmShowDialog(true);
        AVSysOpMultiSelectTemplateContract contract = operation.getDataContractObject();
        Debug::assert(contract != null);

        if (AVMultiSelectionHelper::isMultiSelected(_args))
        {
            contract.setSelectedRecIdSet(AVMultiSelectionHelper::getSelectedRecIDSet(_args));
        }
        else
        {
            operation.AVModifyQueryFromArgs(_args);
        }

        if (operation.startOperation() == SysOperationStartResult::Started) //will return result because we are running synchronous.
        {
            Counter cntr;
            [cntr] = operation.operationReturnValue();
            Info(strFmt("Processed %1 selected records.", cntr));
            //refresh caller here
        }
    }

    public void new(IdentifierName _className = '', IdentifierName _methodName = '', SysOperationExecutionMode _executionMode = SysOperationExecutionMode::Asynchronous)
    {
        super(_className, _methodName, _executionMode);
        
        this.parmClassName(classStr(AVSysOpMultiSelectTemplateService));
        this.parmMethodName(methodStr(AVSysOpMultiSelectTemplateService, runQuery));
        this.parmExecutionMode(SysOperationExecutionMode::Synchronous); //ScheduledBatch: batch job remains visible. ReliableAsynchronous: job is deleted.
        this.parmDialogCaption("AV multi select"); //set dialog caption.
    }

    public static AVSysOpMultiSelectTemplateController construct()
    {
        return new AVSysOpMultiSelectTemplateController();
    }

    public void AVModifyQueryFromArgs(Args _args)
    {
        if (!_args)
        {
            throw error(strFmt("Argument %1 cannot be null.", identifierStr(_args)));
        }

        if (_args.dataset() != tableNum(CustTable))
        {
            throw error(strFmt("Caller buffer must be %1.", tableStr(CustTable)));
        }

        CustTable custTable = _args.record();

        if (custTable.RecId != 0)
        {
            AVSysOpMultiSelectTemplateContract contract = this.getDataContractObject();
            Debug::assert(contract != null);
            if (contract != null)
            {
                Query query = contract.getQuery();
                Debug::assert(query != null);
                if (query != null)
                {
                    QueryBuildDataSource custTable_ds = query.dataSourceTable(tableNum(CustTable));
                    SysQuery::findOrCreateRange(custTable_ds, fieldNum(CustTable, AccountNum)).value(queryValue(custTable.AccountNum));
                    contract.setQuery(query);
                }
            }
        }
    }

    public boolean mustGoBatch()
    {
        boolean ret;
    
        ret = super();
    
        return ret;
    }

    public boolean showQuerySelectButton(str parameterName)
    {
        boolean ret;
    
        ret = super(parameterName);
        AVSysOpMultiSelectTemplateContract contract = this.getDataContractObject();
        Debug::assert(contract != null);

        if (contract.isMultiSelected())
        {
            ret = false;
        }
    
        return ret;
    }

    public boolean showQueryValues(str parameterName)
    {
        boolean ret;
    
        ret = super(parameterName);
        AVSysOpMultiSelectTemplateContract contract = this.getDataContractObject();
        Debug::assert(contract != null);

        if (contract.isMultiSelected())
        {
            ret = false;
        }
    
        return ret;
    }
}


Business logic

public final class AVSysOpMultiSelectTemplateService
{
    //the service method is a class method with a contract as argument. the contract contains the parameters and query (if specified) for the operation.
    //the contract specified here as argument is the contract for our SysOp controller and service classes.
    //this method does not have to return anything but it does return a container to illustrate how to return 'something' from the service method.
    public container runQuery(AVSysOpMultiSelectTemplateContract data)
    {
        Counter         cntr;
        AVTmpRecId      tmpRecId;    //table containing RecIds of the selected records.
        Set             tmpRecIdSet; //set containing the RecIds of the selected records.
        Query           query = data.getQuery();

        if (data.isMultiSelected())
        {
            tmpRecIdSet = data.getSelectedRecIdSet(); //get selected RecIds as a Set.
            AVMultiSelectionHelper::populateTmpRecIdFromSet(tmpRecIdSet, tmpRecId); //populate our temp table from the set that contains the RecIds.
            AVMultiSelectionHelper::modifyQueryAddJoinToTmpTable(query); //modify the query to join with our temp table.
        }

        QueryRun queryRun = new QueryRun(query);

        if (data.isMultiSelected())
        {
            queryRun.setRecord(tmpRecId); //if joined with a temp table, we must set the record on the queryRun object.
        }

        while (queryRun.next())
        {
            CustTable custTable = queryRun.get(tableNum(CustTable));
            info(custTable.AccountNum);
            cntr++;
        }

        return [cntr];
    }

}


Helper class

public final class AVMultiSelectionHelper
{
    public static Set getSelectedRecIDSet(Args _args)
    {
        Set selectedRecIdSet;

        if (!_args)
        {
            throw error(strFmt("Argument %1 cannot be null.", identifierStr(_args)));
        }

        if (!_args.dataset())
        {
            throw error("Must be called with a caller buffer.");
        }

        if (!_args.record().isFormDataSource())
        {
            throw error("The caller buffer must be form datasource.");
        }

        selectedRecIdSet = new Set(Types::Int64);

        MultiSelectionHelper multiSelectionHelper = MultiSelectionHelper::construct();
        multiSelectionHelper.parmDatasource(_args.record().dataSource());

        Common buffer = multiSelectionHelper.getFirst();

        while (buffer.RecId != 0)
        {
            selectedRecIdSet.add(buffer.RecId);
            buffer = multiSelectionHelper.getNext();
        }

        return selectedRecIdSet;
    }

    public static boolean isMultiSelected(Args _args)
    {
        Set selectedRecIdSet = AVMultiSelectionHelper::getSelectedRecIDSet(_args); //arg checks are done in getSelectedRecIDSet().
        return selectedRecIdSet.elements() > 1 ? true : false;
    }

    public static void modifyQueryAddJoinToTmpTable(Query    _query)
    {
        QueryBuildDataSource    qbds;
        SysDictTable            dictTable;
        QueryBuildDataSource    tmpRecId_ds;

        if (!_query)
        {
            throw error(strFmt("Argument %1 cannot be null.", identifierStr(_query)));
        }

        qbds = _query.dataSourceNo(1); //join on first datasource in query.
        dictTable = SysDictTable::newTableId(qbds.table());
        Debug::assert(dictTable != null);

        tmpRecId_ds = qbds.addDataSource(tableNum(AVTmpRecId), tableStr(AVTmpRecId));
        tmpRecId_ds.addLink(dictTable.fieldName2Id(identifierStr(RecId)), fieldNum(AVTmpRecId, RefRecId));
        qbds.clearRanges(); //joined so clear all existing ranges.

        SysQuery::findOrCreateRange(tmpRecId_ds, fieldNum(AVTmpRecId, CreatedBy)).value(curUserId());                                    
        SysQuery::findOrCreateRange(tmpRecId_ds, fieldNum(AVTmpRecId, CreatedTransactionId)).value(queryValue(appl.curTransactionId()));

    }

    public static void populateTmpRecIdFromSet(Set  _recIdSet, AVTmpRecId    _tmpRecId)
    {
        if (!_recIdSet)
        {
            throw error(strFmt("Argument %1 cannot be null.", identifierStr(_recIdSet)));
        }

        SetEnumerator enum = _recIdSet.getEnumerator();

        while (enum.moveNext())
        {
            _tmpRecId.RefRecId = enum.current();
            _tmpRecId.insert();
        }
    }

} 


Sunday 7 March 2021

Microsoft Dynamics 365 Finance - OneBox Virtual Machine - How to download VHD files fast with AzCopy.exe

 

Background

It's possible to download a complete working Microsoft Dynamics 365 Finance virtual machine (VM) called OneBox from the LifeCycle services (LCS) asset library. By the way, this VM will only run with Hyper-V and will not run with other virtualisation software.

This VM can run locally on your companys infrastructure or even on your laptop provided you have a solid state disk (SSD) with enough space and your laptop has enough memory. My Dynamics 365 Finance VM takes up about 120GB of disk space on my external SSD drive.

I enabled Hyper-V on my laptop in Windows features and I am able to run the Dynamics 365 Finance VM with 24GB of RAM allocated to the VM and 8GB RAM for my Windows 10 host. The VM runs okay but it's definitely not suitable for full time development.

When I downloaded the 12 virtual hardisk (VHD) part files for the first time using with my browser, I discovered that it took a long time to download the files and many of the downloads failed. I had to retry them and it was just painful. Each file is about 3 GB. The separate part files, or volumes, will become one large VHD file when executing the first part file which is an executable.

In the LCS in asset library, on each downloadable file, I saw a button with the text "Generate SAS link". When I clicked on it, I got a message that the "SAS link for the asset has been copied to my clipboard and that I can use AzCopy to download the the asset".

I researched AzCopy a bit and downloaded a copy from the Microsoft site to my laptop and extracted it to a folder.

AzCopy can download files much faster than a browser from BLOB storage would because it uses multi threading to download different parts of the file at the same time. A browser uses a single thread per file being downloaded.

You can compare it to a race to empty two identical swimming pools, containing exactly the same amount of water manually just using buckets. With the first swimming pool, there is just one person emptying the pool but with the other, there are ten persons. Not really fair is it? But the ten persons will empty the pool a lot faster than one right?

I selected the first VHD file and clicked on the "Generate SAS link" button. I pasted the link to a text editor to create a command for AzCopy. I  opened a command prompt and downloaded the first file using AxCopy.

To make things easier, I got the idea to add the separate AzCopy command lines in a PowerShell script, one command for each file part . The PowerShell script can download all the part files one after each the other.


Create PowerShell script

  1. Download azcopy and extract it in a folder somewhere like c:\azcopy.
  2. Create a new text file called "downloadvhd.ps1" in the same folder as where azopy.exe is located.
  3. Log in to LCS and go to the Asset Library of your project.
  4. Select the "Downloadable VHD" asset type on the left. If you don't have any part files listed on the right, import them from the Shared Library.
  5. Select the first Part01 VHD file and click on the "General SAS link" button for that file. This will copy the link to  the clipboard.
  6. Open the file created in step 2. with a text editor. On the first line, type: .\azcopy.exe copy '
  7. After the aposthophe, paste the SAS link from the clipboard by pressing Control + V or Paste from the text editor menu. Make sure there is no space between the aposthrophe and the SAS link.
  8. The first part file is an executable file, the rest of the part files are RAR archive volume files with the .rar extension. After the SAS link, type a closing aposthrophe a space and the destination file name e.g.: ' c:\temp\FinandOps10.0.13.part01.exe
  9. For each part file in LCS, create a line in the script file, their file names will be FinandPos10.0.13.part02.rar, FinandPos10.0.13.part03.rar and so on. Important: Click on the "Generate SAS link" button for each file and paste it in the file for the corresponding part file.
  10. When all part files command are in the script file, save it and execute it.

Example PowerShell file

Replace SAS-link-for-Partxx with your own SAS links and change the file names.

.\azcopy.exe copy 'SAS-link-for-Part01' C:\temp\FinandOps10.0.13.part01.exe
.\azcopy.exe copy 'SAS-link-for-Part02' C:\temp\FinandOps10.0.13.part02.rar
.\azcopy.exe copy 'SAS-link-for-Part03' C:\temp\FinandOps10.0.13.part03.rar
.\azcopy.exe copy 'SAS-link-for-Part04' C:\temp\FinandOps10.0.13.part04.rar
.\azcopy.exe copy 'SAS-link-for-Part05' C:\temp\FinandOps10.0.13.part05.rar
.\azcopy.exe copy 'SAS-link-for-Part06' C:\temp\FinandOps10.0.13.part06.rar
.\azcopy.exe copy 'SAS-link-for-Part07' C:\temp\FinandOps10.0.13.part07.rar
.\azcopy.exe copy 'SAS-link-for-Part08' C:\temp\FinandOps10.0.13.part08.rar
.\azcopy.exe copy 'SAS-link-for-Part09' C:\temp\FinandOps10.0.13.part09.rar
.\azcopy.exe copy 'SAS-link-for-Part10' C:\temp\FinandOps10.0.13.part10.rar
.\azcopy.exe copy 'SAS-link-for-Part11' C:\temp\FinandOps10.0.13.part11.rar

Syntax

azcopy.exe copy [source] [destination]

Example:
azcopy.exe copy 'SAS-link-for-Part01' C:\temp\FinandOps10.0.13.part01.exe

Notes

The first part file is an executable file which will extract and combine all the part files into a single VHD file.

You have to check the file names in LCS and adjust the file names  in the script accordingly to match the version in LCS. Don't use my example version.

The SAS link is enclosed in single apostrophies. If the file names contain spaces, then the file name part must also be enclosed with single apostrophies.

Download in progress

Download VHD files from LCS with AzCopy