BOLT Deep Dive slides - HackMD

<style> .reveal section img { margin: 15px 0px; background: none; border: none; box-shadow: none; } </style>  ---  ##### Agenda 1. Short recap: What is BOLT? 1. Fine tuning your BOLT queries (9.0+) 1. Elastic Search support (and other providers) 1. Indexing prices with tax and any number of prices (coming in 9.4) 1. New definition and query syntax (coming in 9.4) ---  ## Short recap: What is BOLT? Fast product retrieval is key on any ecommerce site ---  ## The BOLT Team David Kelemen ![David Kelemen](http://ucommerce.net/media/7248/testdavid.png?width=100px) ![Daniel Berg Frederiksen](http://ucommerce.net/media/7268/meetdanielbf.png?width=100px) Daniel Berg Frederiksen Lukas Vaclavek ![Lukas Vaclavek](http://ucommerce.net/media/7271/meetlukas0.png?width=100px) ![Petr Ilnytsky](http://ucommerce.net/media/7265/meetpetr.png?width=100px) Petr Ilnytsky ---  ## Short recap: What is BOLT? * Fast retrieval & search * Familiar * Extensible ---  ##### Design goals of BOLT ## Fast No need to query the SQL db in product listings Tested on realistic databases ---  ##### Design goals of BOLT ## Extensible Powered by Lucene by default Multiple search providers ---  ##### Design goals of BOLT ## Familiar One interface across providers ```csharp$ Products .Find() .Where(c => c.DisplayName == Match.Fuzzy("shoo")) .OrderBy(c => c.PricesInclTax["EUR"]) .ToList() ``` ---  ## Let's talk "Fast" With BOLT, it's hard to degrade performance, but not impossible First advice: Limit the number of queries ---  ## Fine-tuning your BOLT queries (9.0+) If you do need to optimize a query, we can help! ---  ``` 3644 17:20:42 INFO Running ToList() query on index at: file:///C:/inetpub/wwwroot/sitecore.9.3.0.local/App_Data/Ucommerce/Indexes/A/LuceneDiskIndex-Product-AvenueProductIndexDefinition_7b2c2447f347/en [0 msec] - Where clauses: p => (p.Categories.Contains(value(Ucommerce.Api.CatalogLibrary+<>c__DisplayClass36_0).categoryId.Value) AndAlso (Convert(p.ProductType) != 3)) [0 msec] - Native query: +(+Categories:a2262f1f-e212-4b47-882d-aec58aec8abd -ProductType:3) [1 msec] - Done searching [8 msec] = Hits: 1 of 1 total. [8 msec] = Spent 3 msecs retrieving docs [11 msec] = Spent 4 msecs unwrapping docs [15 msec] LuceneSearch`1.ToList <= CatalogLibrary.GetProducts <= CategoryController.GetProductGuidsInFacetsAndSelectedProductOnSitecoreItem <= CategoryController.Rendering <= ControllerRunner.ExecuteController <= ControllerRunner.Execute <= ExecuteRenderer.Render <= ExecuteRenderer.Process <= CorePipeline.Run <= DefaultCorePipelineManager.Run ``` ---  ### Dissecting a log entry ---  Which index are we using? ``` Running ToList() query on index at: file:///C:/inetpub/wwwroot/sitecore.9.3.0.local/App_Data/ Ucommerce/Indexes/A/ LuceneDiskIndex-Product- AvenueProductIndexDefinition_7b2c2447f347/en [0 msec] ``` ---  Translating the query ``` - Where clauses: p => (p.Categories.Contains(value( Ucommerce.Api.CatalogLibrary+<>c__DisplayClass36_0) .categoryId.Value) AndAlso (Convert(p.ProductType) != 3)) [0 msec] - Native query: +(+Categories:a2262f1f-e212-4b47-882d-aec58aec8abd -ProductType:3) [1 msec] ``` ---  Searching ``` - Done searching [8 msec] = Hits: 1 of 1 total. [8 msec] ``` ---  Retrieving data ``` = Spent 3 msecs retrieving docs [11 msec] = Spent 4 msecs unwrapping docs [15 msec] ``` ---  The mini stack trace ``` LuceneSearch`1.ToList <= CatalogLibrary.GetProducts <= CategoryController.GetProductGuidsInFacetsAndSelectedProductOnSitecoreItem <= CategoryController.Rendering <= ControllerRunner.ExecuteController <= ControllerRunner.Execute <= ExecuteRenderer.Render <= ExecuteRenderer.Process <= CorePipeline.Run <= DefaultCorePipelineManager.Run ``` ---  #### There's more to "Fast" ## The open source Ucommerce Seeder tool ---  ![](https://www.dropbox.com/s/myoys7tbpg0zk09/seeder-asciinema.gif?dl=1) ---  ##### What are the estimated dimensions of your data? ```json { "Products": 500000, "Stores": 3, "CatalogsPerStore": 5, "CategoriesPerCatalog": 100, "AverageVariantsPerProduct": 10, "AverageProductsPerCategory": 1000, "Currencies": 10, "Languages": 7, "PriceGroups": 3000, } ``` ---  Seeding can help you get realistic queries and reach your performance goals before go-live https://github.com/Ucommercenet/Ucommerce-Seeder/ ---  ## Elastic Search provider ##### Motivation: Even higher speed Scalability Maintenance ---  ## Elastic Search provider Just switch provider and re-index Re-use existing queries Option to pass through NEST or JSON queries We aim for v9.4 ---  ## Your search provider? Solr? Azure? Amazon? Google? ...? ---  ### How to make a provider? Right now: Somewhat difficult Future: Easy ---  #### How do I know when I'm done? ### You need a test suite! ---  ### Conclusion It's possible to write your own provider It will be easier in the future: * One test suite across providers * Fewer interfaces ---  ## Facetting prices with tax and any number of prices #### coming in 9.4 ---  ### Before Limited to **one** price field with multiple price groups (by default `Product.UnitPrices`) ---  ### Before ```csharp var productsInCategory = productIndex.Find() .PriceGroup(CatalogContext.CurrentPriceGroup) .Where(facets) .ToFacets(); ``` ---  ### After ```csharp var productsInCategory = productIndex.Find() .Where(facets) .ToFacets(); ``` ---  ## New possibilities ---  ### Before ```csharp class MyIndexDefintion : IIndexDefinition<Product> { public MyIndexDefintion() { ... this.PricesField(p => p.UnitPrices); ... ... } } ``` ---  ### After ```csharp class MyIndexDefintion : IIndexDefinition<Product> { public MyIndexDefintion() { ... this.Field(p => p.UnitPrices).Facet(); this.Field(p => p.PricesInclTax).Facet(); this.Field(p => p.Tax).Facet(); ... ... } } ``` ---  ### Facetting Prices Summary Any number of dictionary type fields supported Not just for monetary amounts :warning: Breaking change -- you will need to change your code a few places Coming in 9.4 ---  ## New definition syntax #### Coming in 9.4 ---  #### Chaining syntax ```csharp this.Field(p => p.Description) .FullText() .DontStore(); ``` ---  #### Chaining syntax ```csharp this.Field(p => p.PricesInclTax["USD 7 PCT VAT"]) .Facet() .DisplayName("Price in $"); ``` ---  #### Range Facets ```csharp this.Field("ShoeSizeEU", typeof(int)) .Facet() .Range(20,27) .Range(27,32) .Range(32,37) .Range(37,42) .Range(42,49) ``` ---  #### Auto Range Facets ```csharp this.Field(p => p.PricesInclTax["EUR"]) .Facet() .AutomaticRanges(5, 10) ``` ---  ##### You've been watching, in order of appearance: ### Use the seeder tool to test your performance before production ### Use the log entries to fine tune your search queries ---  ##### Look forward to ### Elastic Search ### Multiple prices fields with multiple price groups ### Leaner definition and query syntax ---  #### Thank your for your attention! ## Time for questions? ---