Six Key Technology Themes for 2021

For the past few months, I’ve led the EPAM Sitecore Competency Center. It’s been interesting times, to say the least! Much of 2020 was devoted to learning my new role as global head of the Sitecore CC and responding to the Covid-19 pandemic.

Both of these were largely focused internally — but I do hope to write a few posts about some of the interesting accelerators and proofs-of-concept that my fellow EPAMers created during those slow, uncertain months earlier this year. (We’ll showcase several of these at Sitecore Symposium next week.)

But for this post, I’d like to focus on the future, and the 6 themes that will drive digital platforms in 2021:

  1. Decoupled digital systems
  2. Cloud-native architecture
  3. Microservices and API-first development
  4. Optimized digital journeys
  5. Applied testing and analytics
  6. Unified experience profiles

Decoupled digital systems

Decoupling is a strategy in software architecture and code to separate concerns. By splitting features and functions into logically and/or physically separate routines, you can evolve these pieces of a system or program independently. This often improves scalability and maintenance. For content management systems, this means adopting the old idea of the two-stack CMS — treating the management application differently from the content delivery application.

This means you have the ability to tailor experiences for specific devices and for specific purposes. Decoupling also helps reduce the dependencies, which can help speed delivery.

The most extreme form of this can be found in the new wave of “headless” CMS providers, like Contentstack and Contentful. These focus on the management and administration of content through the use of well-defined APIs. But the “head” of the CMS — the delivery side — is all up to you. Sitecore and other CMS platforms are taking a less extreme approach. They provide a head — along with many of the advanced features that come with it — but decouple it from the back-office management application.

Whether you find a truly headless platform or merely a decoupled one a better fit for your needs, it’s clearly the driving theme in digital applications at the moment.

Cloud-native architecture

Many organizations have been surprised to find the benefits of moving to cloud hosting were not what they expected. McKinsey does a great job deconstructing these myths about the cloud. The core insight is that if you treat the cloud simply as “someone else’s datacenter”, you won’t get many of the improvements touted for cloud migration. To reap the benefits, you have to redesign your systems to take advantage of the flexibility that cloud architectures provide. “Lift and shift” is just a first step.

This applies not only to your own organization’s systems, but to many of the software platforms and services you’ve relied on. Most of these will move to a Software-as-a-Service model, and that means designing your systems as loosely connected independent services. That means less code, more configuration, and more integration.

Microservices and API-first development

API-first development underpins my first two themes. By separating concerns into distinct domain services, each with their own API contracts, you improve re-usability and add flexibility to your systems. These small web services can then be scaled and tuned for performance independently of each other.

So the first three trends mutually reinforce each other. Decoupling will lead you to making smaller, more focused applications and services. Standardizing the APIs for these will allow these services to be consumed by and composed into new offerings. And cloud-native architecture will allow these pieces to flex to meet demand and performance targets.

The first three themes were technical; the next three themes have to do with customer experience.

Optimized digital journeys

The recent pandemic underscored the primacy of digital channels for many organizations. And it’s clear that consumers and constituents reward organizations that provide the best digital experiences — through increased sales, positive reviews, and loyalty among other measures. But how can we provide a better digital experience?

The short answer is: figure out what a visitor is trying to do today and make it easy for them to do it. I emphasize the “today” in the last sentence, because often organizations undertake massive data mining operations to study past behavior or try to apply AI or advanced statistical techniques — or perhaps ignore the issue altogether because these projects seem too expensive or complicated! It doesn’t have to be either one of these extremes. It can start with the first question every human customer service representative is trained to ask, “What can I help you with today?”

You can ask this question of your visitors explicitly, or you can use other signals to determine what a visitor is trying to do — like which link an an email the visitor clicked to bring them to the site. Or which social media post they followed. Or what article they landed on through organic search. All of these are useful signals that you can use to help guide and personalize their journey in real time.

And the metrics are also easy to gather. How long did it take them to register an account, check out their cart, or fill out the form? After engaging in one of these journeys, did they respond to your survey? Did they share the article with friends? The chances are, your organization is already collecting much of the information you’d need to optimize the important journeys on your site. Often what organizations lack is time to reflect on these metrics and the know-how required to conduct small experiments. Which leads us to our next theme.

Applied testing and analytics

There’s a big difference between organizations that collect analytics to report status and those that actually apply it to drive improved business outcomes. And 2021 is the year you’ll see the latter kind of organization pull away from the former, due to the disruption caused by the pandemic.

Consumers have had to break a lot of their old habits and adopt new ones. So the companies that can ask smart, targeted questions about critical moments in the customer journey — and then take action on their findings — will distinguish themselves in the marketplace.

There are a range of tools and techniques you can apply. You can do A/B or multivariate tests, measure click-through rates, perform path analysis, conduct focus groups, or provide customer satisfaction surveys. But the tools are less important than being curious and asking interesting questions about the touch-points that serve your customers’ needs.

Unified experience profiles

Once you’ve embraced an experimental mindset and have begun making optimizations, you might be ready for the next step — creating a unified experience profile across all of your digital touch-points.

There are many reasons why a unified, 360-degree view of the customer is beneficial to your organization. One is that it enables you to serve your customers better. For example, your sales team will know what customer support told the customer. When a customer updates profile data, all your systems can see that change. Another is that you can derive new insights about your customers by viewing their behaviors across different channels. And a final reason is that you can better manage sensitive personal information within a comprehensive system than by scattering across numerous back-office systems.

This is a tall order, given the number of functions within your enterprise that rely on this data. Implementing a comprehensive view of customer data is a multi-year effort. But what makes this a powerful theme for 2021 is both the pandemic, which has emphasized digital remote work and de-emphasized paper and face-to-face transactions, as well as privacy and confidentiality regulations worldwide.

Many organizations are well on their way to implementing a 360-view. But harnessing its benefits — not simply reacting to events — will lead to many interesting opportunities.

Summing up

These 6 themes form the new digital playbook that many leading-edge organizations follow as they launch and scale their enterprises. The enterprise solutions I’m working on now involve many, if not all, of these themes.

We’ll be talking about these themes at Sitecore Sympsoium. I’m curious about whether these themes will resonate with others in the Sitecore community and in the wider digital ecosystem.

Sitecore MVP 2020

I’m glad to be part of the Sitecore MVP community again! Sitecore announced their MVPs for 2020 last week, so I’m once again a man with a badge: a Sitecore Commerce badge.

This is the second time I’ve been a Sitecore MVP, my first award having been for Commerce in 2018. This year, I’ll be continuing my work with the Sitecore Commerce – Microsoft D365 Retail connector as well as getting involved in my company’s Sitecore Commerce – Hybris connector.

In addition to the ERP connectors we’ve been building, EPAM also released its headless commerce accelerator as open source just before Sitecore Symposium. I think there’s a lot of potential to build on each of these efforts.

EPAM had 14 awardees this year, so I’m in good company. (In both senses of the word!) But I’m especially pleased that my co-presenter of the D365 Connector at Symposium 2019, Vsevolod (Seva) Kolonistov, is also a technology MVP for 2020 — the only one in Russia, and the organizer of the St. Petersburg Sitecore User Group.

It’s going to be an exciting 2020!

Introducing the Sitecore Commerce 9 Connector for Microsoft D365

At Sitecore Symposium 2019 this year, I collaborated with my coworker and friend Vsevolod Kolonistov to present the new Sitecore Commerce 9 connector for Microsoft D365 Retail.

We’d gained a lot of experience working with the prior version of the connector, so when it came time to modernize it to use the new Sitecore Commerce 9 infrastructure, we had a lot of advice and knowhow to share with the Sitecore Commerce team. We also wanted to share it with the rest of the Sitecore community, and where better to do that than at Symposium?

Sitecore Commerce and Microsoft D365 Retail are an interesting combination. While Sitecore Commerce alone is more than capable of powering B2C ecommerce websites right out of the box, if you also need to worry about physical points-of-sale or call center operations, you’ll need to pair it with another enterprise-grade tool. Which is where Microsoft D365 Retail comes in.

The integration is an interesting mix:

  • Batch operations for syncing catalog information and product media 
  • Real-time operations for cart, checkout, and customer accounts

But I won’t go into all the details here — you can get the slides below.

Yes, Symposium was such a whirlwind that I’m only now posting the slides from the breakout session!

Presenting the Sitecore-D365 connector at Symposium 2019
Dean Thrasher (left) and Vsevolod Kolonistov (right) presenting the new Sitecore Commerce 9 connector for Microsoft D365 Retail

One thing I neglected to mention in the slides themselves, though it was covered in the Q&A session, is that the D365 Connector isn’t available from the Sitecore developer site directly. For now, if you’re interested, you’ll need to contact your Sitecore representative directly.

I’m curious to see how many other companies pursue this sort of integration. D365 Retail — a specialization of D365 for Finance and Operations — is still evolving rapidly. Microsoft wants it to be a enterprise-grade SaaS ERP, but you can still see a lot of the old Dynamics AX client-server architecture peeking through in places. I expect it to improve with time, though, so perhaps we’ll see more of this Sitecore-D365 combination in the future. If so, we’ve got the connector ready for it!

Sitecore Symposium 2019 Customer Showcase

Every year at Sitecore Symposium, Sitecore features customer success stories. This year, I had the honor to co-present the relaunch of the Lowe’s Canada website with Tanbir Grover, VP of Digital and Omnichannel at Lowe’s Companies Canada.

Sitecore Symposium 2019 customer showcase
On the big stage at Sitecore Symposium 2019

In August 2019, after nearly 3 years of work (if you include the the first proof-of-concept in 2016), the new website went live across Canada. The results so far have been very promising, especially for mobile ecommerce. Tanbir was able to share some impressive statistics. More important than the numbers, though, is the improvements in flexibility and agility. The retail sector has been reinventing itself, and now the digital team at Lowe’s Canada can respond to changing trends and innovate in key areas such as content marketing, personalization, and search.

I wish we’d been able to put the whole team on stage — from Lowe’s Canada and EPAM. This project would not have succeeded without a lot of very smart people from around the world solving difficult commerce problems at scale. I’ve very proud to have been a part of that team.

Sitecore Hackathon 2019

This was my first year participating in Sitecore Hackathon. It’s an annual event that brings teams from around the world together to create a useful module, plugin, or code example for the latest version of the Sitecore platform.

Although it’s been going on for several years, I always find myself in the middle of projects at Hackathon time, and the idea of staying up for 24 hours straight to do more Sitecore seemed a bit hardcore to me. This year was no exception, with Hackathon falling between two week-long international trips.

But I resolved to give it a shot anyway. I’d heard so many good things about it from the Sitecore community, that with a little prodding from my coworkers, I took the plunge and registered for Hackathon 2019.

Here’s what I learned:

  • Come prepared with ideas. You don’t know what themes will be selected in a hackathon — those are announced 1 hour prior to the start of the event. So it pays to have a few alternatives.
  • Make sure your ideas are feasible for a small team in one day. Respect the “minimum” in minimum viable product!
  • Make sure you have your baseline Sitecore instance ready to go before Hackathon begins. You won’t want to waste precious hours reading the installation guide and downloading modules during the event.
  • As with real projects, co-located teams have it easier than distributed ones. You can keep each others’ energy going, communicate freely, and use that precious screen real estate for something other than a web meeting!
  • Make a schedule. Know when your teammates are going to be available. Remember to allow enough time to commit the code, write the documentation, and record the video.
  • Have fun with it! This is an extracurricular activity, so play around and take some risks!

For my first time, I messed up the preparation step and was far too ambitious about what we could deliver with a small team. But it was still fun to hang out online. It also reminded me how much I like writing code. I’ve been doing high-level architecture for the past two years, and I’ve really missed building things out of bits and bytes.

That may be the most important lesson from Hackathon this year — not to let another year go by without programming something fun.

 

Sitecore and StyleLabs

The big announcement at Sitecore Symposium this year was Sitecore’s acquisition of StyleLabs, a marketing technology company that provides several software-as-a-service tools:

  • Digital Asset Management (DAM)
  • Marketing Resource Management (MRM)
  • Product Information Management (PIM)
  • Digital Rights Management (DRM)

It wouldn’t be enterprise software without all the acronyms, would it?

Sitecore aquires StyleLabs
Sitecore CEO Mark Frost talks with StyleLabs’ co-founders at Sitecore Symposium 2018

This announcement was a big deal for Sitecore, because its current tool for handling media assets — the Sitecore Media Library — was increasingly showing its age. The structure and features of the Media Library haven’t changed much in 10 years. It’s one of the first things that partners needed to extend or replace for any company that was serious about managing digital media. And it had nothing to offer in terms of workflow tools apart from the simple “publishing approval” style of workflow used for web pages.

There were a few go-to solutions for DAM integrations — both Digizuite and ADAM Software (now part of Aprimo) have decent integrations with software. For any of the other capabilities listed above, you most likely would need to build your own solution.

So customers and partners were really pleased that Sitecore will have something first-party to offer in these areas. If the schedule holds, StyleLabs integration as a plug-in within by the end of 2018, with a full integration planned in the first half of 2019.

More than a product line extension?

While addressing a gap in its offering is important — especially since Sitecore often goes head-to-head with Adobe Experience Manager in big deals — I’m more excited about the long-term potential.

This acquisition brings some new technologies, new approaches, and new team members into the Sitecore family.

StyleLabs was founded in 2011, and its cloud-hosted, SaaS offerings reflect a later technology generation than Sitecore,  whose roots go back before the turn of the millennium. (It says a lot about the strength of Sitecore’s fundamentals that it has lasted this long!) As Sitecore modernizes its architecture stack, having the StyleLabs’ expertise in SaaS technologies and business model will be invaluable.

While Sitecore was always great about storing and managing content and digital assets, in really didn’t address the production of those assets in the first place. StyleLabs understands the creative workflow that goes into making these digital artifacts, and I hope to see their knowledge reflected in future versions of the Sitecore platform.

I also expect a strong of cross-pollination of ideas. Sitecore and StyleLabs execute similar concepts in different ways. For example, Sitecore structures information in a hierarchy or tree, which works extremely well for managing URLs in a website. StyleLabs takes a relational approach to content, which offers more flexibility but perhaps less consistency.

So there are a lot of cues — from software architecture to user interface — that these teams can take from each other. While filling gaps in the product line may boost the short-term, I’m really curious what this means for Sitecore over the long term.

 

 

 

Sitecore Symposium and MVP Summit 2018

I’ve just returned from the Sitecore Symposium 2018 in Orlando, FL. It’s been two years since I attended the last one (in New Orleans). My, how it’s grown! There were more than 3000 attendees at Disney’s Swan and Dolphin, with breakout sessions divided into 5 different tracks, ranging from marketing, to technology, to getting started with the platform. It was an intense 3 days of sessions and networking.

There’s so much to talk about, I think I’ll need to break it up into a series of posts over the next few weeks. But I’ll start with how thankful I am to have attended the MVP Summit in the day and half after the closing Symposium keynote.

Symposium_2018_Badge

This was my first year as a Sitecore Commerce MVP, so I’d never been to the MVP Summit. I’d heard great things about it from lots of other MVPs. It’s an opportunity for Sitecore insiders to share upcoming preview releases and talk candidly with community leaders about what’s coming next. It’s also a chance for MVPs to provide face-to-face feedback to Sitecore employees about what’s working well and what can be improved. And it’s naturally a time for MVPs to network with each other.

Part of what makes the MVP Summit work is that it has a smaller number of attendees — about 300 — but a wide diversity of backgrounds. There were MVPs from around the world, some from implementation partners, some from Sitecore’s customers, and some from makes of tools and extensions that work with Sitecore. Many of these were names I recognized from their blog articles, Slack chats, or forum posts. It was great to finally match faces with the names.

Another part of what makes the Summit work is that there’s a huge diversity of topics covered, from Commerce (my focus at the moment) to developer tools, to strategy, to documentation, to the operation of the Sitecore MVP program itself. Some of the best sessions and conversations I participated in were the ones I hadn’t planned on attending.

Finally, there’s a diversity of formats, from presentations, to Q&A sessions, roundtables, and even a “special event” where MVPs can just hang out together and swap stories. Symposium itself tends to either be large presentations or the scrum within the booths at the partner pavilion. So it was nice to have sessions that were less formal and less pressured — I certainly got more out of it than I anticipated. And that’s despite my expectations having been raised by several other MVPs.

So now my task is to figure out how to stay in the club next year!

Sitecore Solr Cloud Support

On the roadmap for future releases of Sitecore 9 is true SolrCloud support. This has been a sticking point with many scaled implementations of Sitecore since SolrCloud was first introduced in 2013. For the most part, Sitecore implementations have relied upon either the Solr Master-Slave model to ensure high availability and load balancing (at least for queries) or have muddled through with Solr Cloud approximations.

A history lesson

To understand where Sitecore stands with regard to Solr Cloud support, you have to know a little history behind how search was implemented in Sitecore. There are two mechanisms for finding items in Sitecore. The first is to use Sitecore’s database query mechanisms, which rely on an XPath-like syntax for traversing the content tree. This is a slow method, but is used for simple queries to traverse parent-child relationships and is often used with Sitecore’s field types that display hierarchical relations: the treelist, droplist, and droptree, for example.

The second method is to use a full text search provider. Back in the Sitecore 4 or 5 days, this was dtSearch, but Lucene quickly became the default full text search provider, due to the extensive documentation and examples available. Like dtSearch, Lucene was a full-text indexing engine meant to be used in embedded applications. These tools worked great on a developer machine, or on a single-sever “all-in-one” demo configuration, but caused issues when deploying into a multi-server environment. Since each server maintained its own index, issues with out-of-sync indexes were extremely common.

Solr, an extension of the Apache Lucene project, was introduced to solve this scalability issue. It offered an enterprise platform that used the Lucene API and supported many of the extensions and plugins developed by that community, and allowed indexing and search functions to run on a separate instance. Since the API was largely compatible with Lucene, it was easy for Sitecore and other CMS platforms to transition to Solr.

Solr scalability models

The early approach to Solr scalability was the master-slave model. One server in a Solr cluster — the master — performs all the write operations, but any server in the cluster can respond to queries. This approach is one of eventual consistency for read operations. If a slave fails, the load can be redistributed to the other members of the cluster. If the master fails, however, no new write operations can be performed until one of the slaves can be reconfigured to act as the new master.

Master-slave was a challenge for Sitecore for two reasons.

  1. Sitecore assumes that its indexing and search server shares a single URL. But really, it should know of two URLs: the load-balanced URL used for reads, and the single URL that points to the primary Solr server for writes.
  2. To recover from a master failure, manual intervention was required to promote a slave to a master. This requires a reboot of the slave Solr server, which is why a true high-availability Solr master-slave arrangement should really have been called a “master-slave-slave” configuration, and needs to contain a minimum of three instances.

To address the first problem, you need to configure your load balancer to spread all GET requests across all of your master and slave instances, but to route POST requests only to the master server. Or you need to modify the Sitecore code and configuration to separate writes from reads, and send each to a different URL. (The second approach may be required if your query parameters become so complex that you run into the GET request character limit for the query string.)

To address the second problem — which was an issue for all systems using Solr, not just Sitecore — the Solr project introduced SolrCloud.

SolrCloud has nothing to do with cloud computing, despite its name. You can have on-premise datacenter deployments of SolrCloud. It’s just the name given to the new high availability and scalability model that replaces the old master-slave-slave approach.

In a SolrCloud, ZooKeeper nodes are used to determine which Solr instance acts as the primary instance responsible for write operations. If the primary fails, one of the secondaries is promoted to act as the new primary automatically. There may be a few write errors as the problem gets detected, a new election of a primary occurs, and the new primary takes over write responsibilities, but this is generally a short interval. No manual intervention is required.

This is a great approach, except for one problem: Sitecore (and the Solr libraries it depends on) don’t understand ZooKeeper.

Who’s in charge of this zoo?

In the Java world, the Solr4j library has had support for ZooKeeper “baked-in” so that clients don’t have to figure out which Solr node is the primary. The CloudSolrClient in the Solr4j library client communicates with the Zookeeper nodes to discover Solr endpoints. For write operations, it helps determine which Solr instance will handle the write requests, and all index update operations are transparently sent to that instance. For read operations, the CloudSolrClient uses the LBHttpSolrClient class as a software load balancer. There’s no additional work required!

But Sitecore is a Microsoft.NET application, and its ContentSearch API ultimately relies upon a library called Solr.NET. Solr.NET is not maintained by the Apache Solr project itself, but by a group of open source developers. As a result, Solr.NET support for Solr versions often lag behind that of the Java client libraries. This is why Sitecore only supports up to the Solr 6 series, even though Solr 8 will be released very soon.

And for a long time, Solr.NET didn’t fully support Solr 6 — which is why it didn’t have the CloudSolrClient baked in, or any knowledge of how to communicate with Zookeeper. But as of January 2018, developers can finally get a version of Solr.NET with support for SolrCloud and versions up to Solr 7. There’s even a SolrNet.Cloud NuGet package.

Now all we have to do is wait for Sitecore to incorporate it into the .NET platform. What about it Sitecore? Can we have it in time for Symposium this year? Or at least by Christmas?

 

Sitecore MVP 2018

Dean Thrasher is a Sitecore Commerce MVP for 2018

I was pleased and excited to be named a Sitecore Commerce MVP for 2018! While I have worked with Sitecore for many years, this is the first time I have earned an MVP award.

The official announcement is here: https://www.sitecore.com/company/press-and-media/press-releases/2018/01/sitecore-names-its-2018-most-valuable-professionals

Typically, I’m so busy with client implementations, I don’t have much time to focus on social media and knowledge sharing, apart from my work with the DC Sitecore User Group. But during 2017, I had the opportunity to work closely with a major retail client and the Sitecore product development team for their Sitecore Connect for Microsoft Dynamics 365 module. It’s been an awesome experience working directly with Sitecore on fast-moving Microsoft tech in the Commerce space.

For more details about the Sitecore MVP program, and a breakdown of statistics regarding the awardees for this year, see: https://www.sitecore.com/company/blog/521/announcing-the-2018-sitecore-mvp-awards-4508

SCpbMD Catalog Sync Explained

I’ve been meaning to post this diagram for a while. I’ve used this to explain the Sitecore Commerce catalog data sync operation to at least three different clients since I drafted it this summer. And although I created it for the Microsoft Dynamics 365 version of the connector, it’s similar for Dynamics AX and other PIM systems as well.

SCpbMD_DataSync_2017-11-27.png

In the D365 box, you have the UI application, which admins can use to publish catalog data once it has been validated. There are a lot of elements that must be configured an working correctly to have a valid catalog, but a few of the key pieces are:

  • An online navigation hierarchy for your online channel
  • An assortment for your online channel containing released products
  • A catalog associated with the online channel
  • Products assigned to nodes on the online navigation hierarchy
  • Product attributes defined and attached to nodes on your online navigation hierarchy

Once the catalog is published, it’s ready to go as far as the “headquarters” database is concerned. But in Dynamics AX / D365, it also has to be distributed — sent to the online channel database using distribution jobs.

Once the catalog is in the online channel database, the D365 Retail Server can read from it. External applications can read catalog data from the online channel database using the Retail Server APIs, a curious mix of web services that aren’t quite WCF and aren’t quite REST.

This is where the Sitecore part of the picture comes into play. Sitecore provides a sample console application that uses Sitecore’s Data Exchange Framework to fetch data from the Dynamics Retail server. It transforms it into an XML file that can then be imported into Sitecore’s Commerce Server. (Sitecore 9 also uses this catalog.xml file format, though the old commerce server components are no longer used.)

This places the product and category definitions and data into the product catalog database. This product catalog database acts as an “edge cache” that keeps just the products the site will use close to the infrastructure of the website itself. It provides some redundancy in case that communications problems occur between Sitecore and D365.

The last step in the process is the catalog data provider. Sitecore XP uses a data provider to access the product catalog database, creating virtual Sitecore items that appear within the Sitecore UI. Product and category data are not stored as “real Sitecore items” in the sense that they live in the standard Sitecore master or web databases.

Watch the arrows!

Note the color and direction of the arrows in the diagram above. The orange arrows are the ones controlled by the Sitecore console app and Data Exchange Framework. The arrows in blue are either part of standard D365 functionality or belong to extensions to the Sitecore platform. The orange arrows could have been labeled “Extract, Transform, and Load” because that’s exactly the operations performed by the catalog sync. (If I ever redraw the diagram, I might update the labels to say just that!)

The direction of the arrows are important, too. Catalog information must be sent from AX HQ to the channel database by those batch distribution jobs. If those jobs aren’t running, then no updates occur in the channel DB, and no updates will be returned in by the Retail Server API.

Once the data is available at the Retail Server, it’s up to the Sitecore catalog sync process to fetch the latest data from the Retail Server. This can be run manually or as a scheduled job, but note that there are no notifications here — D365 doesn’t push the data to Sitecore, Sitecore pulls the data when it needs to.

A sequence of batches

This is definitely not a real-time process. As you might imaging from the number of batches, pushes, and pulls shown in the diagram, it can take a significant amount of time to move an update — like a adding a new product to the catalog or setting up a new product attribute for a category — from D365 HQ into Sitecore XP. If every batch job involved executed on a 15 minute timer, it could take 45-60 minutes for that product to appear on the site. The interval could be longer depending on the size of the catalog and the number of changes made.

There’s more than one way to do it

Although Sitecore provides the catalog sync code as part of its commerce connectors, it’s really just an example or starter kit for us to use. In practice, you’ll need to modify the logic used to generate the catalog.xml file to import into Sitecore. You may also need to move the data sync process to other servers for scalability or performance reasons. Or you could replace Sitecore’s Data Exchange Framework with another ETL framework or a business process orchestration suite like BizTalk.

The connector is just a starting point for implementation, and hopefully the diagram and my explanation of it makes a good starting point for discussion with your team or client about how the process of syncing catalogs might work.