Last night, our latest Market Package with enhancements, fixes and improvements was released. It contains one new API feature, as well as a vast improvement of the speed of another.
As for the new feature, it is now possible to login (‘Connect‘ in API terms) as an administrator. This facilitates integration with external accounting applications, e.g. reporting tools.
To utilize this feature, instead of ‘logging in’ by using the Connect() method, you use the ConnectAsAdministrator() method, which takes the administrator user’s credentials (administrator agreement number, user ID and password), as well as the agreement number of a client to which the administrator user has access, as parameters. Note that the client in question of course still needs to have the API add-on module enabled.
As for optimizations, we were recently made aware of some serious bottlenecks in our GetDataArray() methods when large registers are downloaded. Somewhat embarrassing, since we’re constantly pushing the data array methods as the recommended way of bulk reading/writing of data 🙁
After digging deeper into this, we discovered that the culprit was our ORM tool’s extremely conservative approach to checking for ‘dirty objects’ in its cache. While you’d expect the execution time for a GetDataArray() call to be proportional to the number of objects retrieved, in practice, it turned out to be polynomial instead! Not exactly what you’d want when downloading product registers in the thousands…
While we could conceivably implement a generic workaround directly in our ORM tool, for the sake of risk minimizing, we have instead opted to implement bespoke optimizations on a per-object level – starting with the objects our logs show us to be the most frequently used.
Last night’s release includes this optimization for the IProduct object – in practice, the Product_GetDataArray() method, which now handles upwards of a couple of thousand objects in a few seconds. The relative speed improvement increases with the number of objects – we’ve measured the speed increase to a factor of 10+ with 2,000 products.
Our next Market Package release, due in mid-May, will include similar optimizations for a lot of other objects (IDebtor, ICreditor, IAccount, IInvoice, IInvoiceLine, etc.). As for all these registers, we still recommend ‘paging’ to approximately 500 objects for each call to its GetDataArray() method.
For more detailed technical API documentation, take a look here.
Hi. Nice to know that you are working on the perfomance of this. When we implemented our integration client to your system, we experienced the problem and actually measured the most efficient page size (time / item) to be around 80.
We will try again in may when IAccount and ICreditor have been fixed.
Btw, does this update require DLL upgrade or is it all server side?
Hi Magnus,
Admittedly, I haven’t tried the more-round-trips-vs-smaller-page-sizes measuring approach – but for 80 to be the optimal page size sounds fairly small to me. Then again – while 80 vs 500 yields more than 8 times as many round trips, the difference is likely negligible when weighed against the server-side speedup of smaller pages.
As for DLL updates, that is only necessary if you wish to use a new function – in this case the ConnectAsAdministrator() function. The product data array optimization is purely server-side, so everyone will benefit from that automagically 🙂
Best regards,
Christian Estrup
e-conomic