{"id":646,"date":"2017-08-16T22:16:34","date_gmt":"2017-08-16T20:16:34","guid":{"rendered":"https:\/\/qappdesign.com\/code\/?p=646"},"modified":"2017-11-15T07:44:34","modified_gmt":"2017-11-15T05:44:34","slug":"paging-mongodb-avoid-poor-performance","status":"publish","type":"post","link":"https:\/\/qappdesign.com\/code\/paging-mongodb-avoid-poor-performance\/","title":{"rendered":"Paging in MongoDB &#8211; How to actually avoid poor performance ?"},"content":{"rendered":"<p>What is the best way (performance wise) to paginate results in MongoDB ? Especially when you also want to get the total number of results ?<br \/>\n<em>Project running with .NET Core 2.0<\/em><\/p>\n<h4>Where to start ?<\/h4>\n<p>For answering to these questions, let&#8217;s start from the datasets defined in my earlier article <a href=\"https:\/\/qappdesign.com\/code\/search-best-places-mongodb-linq-netcore\/\">Part 1: How to search good places to travel (MongoDb LINQ &#038; .NET Core)<\/a>. That article was quick introduction, on how to load big chunks of data and then retrieve values using WebApi and LINQ. Here, I will start from that project, extending it with more details related to paging the query results. You could also check <a href=\"https:\/\/qappdesign.com\/code\/mongodb-and-linq-how-to-aggregate-and-join-collections\/\">Part 3 &#8211; MongoDb and LINQ: How to aggregate and join collections<\/a><\/li>\n<p>You can find the full solution, together with the data here: <a href=\"https:\/\/github.com\/fpetru\/WebApiQueryMongoDb\" target=\"_blank\">https:\/\/github.com\/fpetru\/WebApiQueryMongoDb<\/a> <\/p>\n<h4>Topics covered<\/h4>\n<ul>\n<li>Paging query results with skip and limit<\/li>\n<li>Paging query results using last position<\/li>\n<li>MongoDb BSonId<\/li>\n<li>Paging using MongoDb .NET Driver<\/li>\n<\/ul>\n<h4>To install<\/h4>\n<p>Here are all the things needed to be installed:<\/p>\n<ul>\n<li><a href=\"http:\/\/visualstudio.com\/free\" target=\"_blank\">Visual Studio Community 2017<\/a>, including .NET Core option<\/li>\n<li><a href=\"https:\/\/www.mongodb.com\/download-center#community\" target=\"_blank\">MongoDB<\/a> and <a href=\"https:\/\/robomongo.org\/\" target=\"_blank\">Robomongo<\/a><\/li>\n<\/ul>\n<h4>See the results<\/h4>\n<p>Here are few steps to have the solution ready, and see the results immediately:<\/p>\n<ol>\n<li>Clone or <a href=\"https:\/\/github.com\/fpetru\/WebApiQueryMongoDb\/archive\/master.zip\" target=\"_blank\">download the project<\/a><\/li>\n<li>Run <em>import.bat<\/em> file from Data folder &#8211; this will create the database (TravelDb), and fill in two datasets<\/li>\n<li>Open solution with Visual Studio 2017 and check the connection settings <em>appsettings.json<\/em><\/li>\n<li>Run the solution<\/li>\n<\/ol>\n<p>If you have any issues on installing MongoDb, setting up the databases, or project structure, please review my earlier <a href=\"\/search-best-places-mongodb-linq-netcore\/\">article<\/a>. <\/p>\n<h4>Paging results using cursor.skip() and cursor.limit()<\/h4>\n<p>If you do a Google search, this is usually the first presented method to make pagination of the query results in MongoDB. It is a straightforward method, but also expensive in terms of performance. It requires the server to walk from the beginning of the collection or index each time, to get the offset or skip position, before actually begin to return the result you need. <\/p>\n<p>For example: <\/p>\n<pre>\r\ndb.Cities.find().skip(5200).limit(10);\r\n<\/pre>\n<p>The server will need to parse the first 5200 items in WikiVoyage collection, and then return the next 10. This doesn&#8217;t scale well due to <em>skip()<\/em> command. <\/p>\n<h4>Paging using the last position<\/h4>\n<p>To be faster, we should search and retrieve the details starting from the last retrieved item. As an example, let&#8217;s assume we need to find all the cities in France, with a population greater than 15.000 inhabitants.<\/p>\n<p>Following this method, the initial request to retrieve first 200 records would be:<\/p>\n<p><strong>LINQ Format<\/strong><br \/>\nWe first retrieve <em>AsQueryable<\/em> interface:<\/p>\n<pre>\r\nvar _client = new MongoClient(settings.Value.ConnectionString);\r\nvar _database = _client.GetDatabase(settings.Value.Database);\r\nvar _context = _database.GetCollection&lt;City&gt;(&quot;Cities&quot;).AsQueryable&lt;City&gt;();\t\r\n<\/pre>\n<p>and then we run the actual query:<\/p>\n<pre>\r\nquery = _context.CitiesLinq\r\n                .Where(x =&gt; x.CountryCode == &quot;FR&quot;\r\n                            &amp;&amp; x.Population &gt;= 15000)\r\n                .OrderByDescending(x =&gt; x.Id)\r\n                .Take(200);\r\n\t\t\t\t\r\nList&lt;City&gt; cityList = await query.ToListAsync();\r\n<\/pre>\n<p>The subsequent queries would start from the last retrieved <strong>Id<\/strong>. Ordering by BSonId we retrieve the most recent records created on the server before the last Id. <\/p>\n<pre>\r\nquery = _context.CitiesLinq\r\n                .Where(x =&gt; x.CountryCode == &quot;FR&quot;\r\n                         &amp;&amp; x.Population &gt;= 15000\r\n                         &amp;&amp; x.Id &lt; ObjectId.Parse(&quot;58fc8ae631a8a6f8d000f9c3&quot;))\r\n                .OrderByDescending(x =&gt; x.Id)\r\n                .Take(200);\r\nList&lt;City&gt; cityList = await query.ToListAsync();\r\n<\/pre>\n<h4>Mongo&rsquo;s ID<\/h4>\n<p>In MongoDB, each document stored in a collection requires a unique _id field that acts as a primary key. It is immutable, and may be of any type other than an array (by default a MongoDb ObjectId, a natural unique identifier, if available; or just an auto-incrementing number). <\/p>\n<p>Using default ObjectId type,<\/p>\n<pre>\r\n[BsonId]\r\npublic ObjectId Id { get; set; }\r\n<\/pre>\n<p>it brings more advantages, such as having available the <em>date<\/em> and <em>timestamp<\/em> when the record has been added to the database. Furthermore, <em>sorting<\/em> by ObjectId will return last added entities to the MongoDb collection.<\/p>\n<pre>\r\ncityList.Select(x =&gt; new\r\n\t\t\t\t\t{\r\n\t\t\t\t\t\tBSonId = x.Id.ToString(), \/\/ unique hexadecimal number\r\n\t\t\t\t\t\tTimestamp = x.Id.Timestamp,\r\n\t\t\t\t\t\tServerUpdatedOn = x.Id.CreationTime\r\n\t\t\t\t\t\t\/* include other members *\/\r\n\t\t\t\t\t});\r\n<\/pre>\n<h4>Returning fewer elements<\/h4>\n<p>While the class <em>City<\/em> has 20 members, it would be relevant to return just the properties we actually need. This would reduce the amount of data transferred from the server.<\/p>\n<pre>\r\ncityList.Select(x =&gt; new\r\n\t\t\t\t\t{\r\n\t\t\t\t\t\tBSonId = x.Id.ToString(), \/\/ unique hexadecimal number\r\n\t\t\t\t\t\tName,\r\n\t\t\t\t\t\tAlternateNames,\r\n\t\t\t\t\t\tLatitude,\r\n\t\t\t\t\t\tLongitude,\r\n\t\t\t\t\t\tTimezone,\r\n\t\t\t\t\t\tServerUpdatedOn = x.Id.CreationTime\r\n\t\t\t\t\t});\r\n<\/pre>\n<h4>Indexes in MongoDB &#8211; few details<\/h4>\n<p>We would rarely need to get data, in exact order of the MongoDB internal ids (_id)I, without any filters (just using <em>find()<\/em>). In most of the cases, we would retrieve data using filters, and then sorting the results. For queries that include a sort operation without an index, the server must load all the documents in memory to perform the sort before returning any results.<\/p>\n<p><strong>How do we add an index ?<\/strong><br \/>\nUsing RoboMongo, we create the index directly on the server:<\/p>\n<pre>\r\ndb.Cities.createIndex( { CountryCode: 1, Population: 1 } );\r\n<\/pre>\n<p><strong>How do we check our query is actual using the index ?<\/strong><br \/>\nRunning a query using <em>explain<\/em> command would return details on index usage:<\/p>\n<pre>\r\ndb.Cities.find({ CountryCode: \"FR\", Population : { $gt: 15000 }}).explain();\r\n<\/pre>\n<p><img src=\"data:image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\" data-src=\"https:\/\/qappdesign.com\/code\/wp-content\/uploads\/2017\/06\/mongodb-index-describe.jpg\" alt=\"\" class=\"lazyload \" \/><noscript><img src=\"https:\/\/i0.wp.com\/qappdesign.com\/code\/wp-content\/uploads\/2017\/06\/mongodb-index-describe.jpg?w=960&#038;ssl=1\" alt=\"\" data-recalc-dims=\"1\" \/><\/noscript><\/p>\n<p><strong>Is there a way to see the actual query behind the MongoDB LINQ statement ?<\/strong><br \/>\nThe only way I could find this, it was via <em>GetExecutionModel()<\/em> method. This provides detailed information, but inside elements are not easy accessible.<\/p>\n<pre>\r\nquery.GetExecutionModel();\r\n<\/pre>\n<p>Using the debugger, we could see the elements as well as the full actual query sent to MongoDb.<br \/>\n<img src=\"data:image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\" data-src=\"https:\/\/qappdesign.com\/code\/wp-content\/uploads\/2017\/08\/mongodb-linq-query.jpg\" class=\"lazyload \" \/><noscript><img src=\"https:\/\/i0.wp.com\/qappdesign.com\/code\/wp-content\/uploads\/2017\/08\/mongodb-linq-query.jpg?w=960&#038;ssl=1\" data-recalc-dims=\"1\" \/><\/noscript><br \/>\nThen, we could get the query and execute it against MongoDb using RoboMongo tool, and see the details of the execution plan.<\/p>\n<h4>Non LINQ way &#8211; Using MongoDb .NET Driver<\/h4>\n<p>LINQ is slightly slower than using the direct API, as it adds abstraction to the query. This abstraction would allow you to easily change MongoDB for another data source (MS SQL Server \/ Oracle \/ MySQL etc.) without many code changes, and this abstraction brings a slight performance hit.<\/p>\n<p>Even so, newer version of the MongoDB .NET Driver has simplified a lot the way we filter and run queries. The fluent interface (IFindFluent) brings very much with LINQ way of writing code.<\/p>\n<pre>\r\nvar filterBuilder = Builders&lt;City&gt;.Filter;\r\nvar filter = filterBuilder.Eq(x =&gt; x.CountryCode, &quot;FR&quot;)\r\n\t\t\t\t&amp; filterBuilder.Gte(x =&gt; x.Population, 10000)\r\n\t\t\t\t&amp; filterBuilder.Lte(x =&gt; x.Id, ObjectId.Parse(&quot;58fc8ae631a8a6f8d000f9c3&quot;));\r\n\r\nreturn await _context.Cities.Find(filter)\r\n\t\t\t\t\t\t\t.SortByDescending(p =&gt; p.Id)\r\n\t\t\t\t\t\t\t.Limit(200)\r\n\t\t\t\t\t\t\t.ToListAsync();\r\n<\/pre>\n<p>where <em>_context<\/em> is defined as<\/p>\n<pre>\r\nvar _context = _database.GetCollection&lt;City&gt;(&quot;Cities&quot;);\t\r\n<\/pre>\n<h4>Implementation<\/h4>\n<p>Wrapping up, here is my proposal for the paginate function. <em>OR<\/em> predicates are supported by MongoDb, but it is usually hard for the query optimizer to predict the disjoint sets from the two sides of the OR. Trying to avoid them whenever is possible is a known trick for query optimization. <\/p>\n<pre>\r\n\/\/ building where clause\r\n\/\/\r\nprivate Expression&lt;Func&lt;City, bool&gt;&gt; GetConditions(string countryCode, \r\n\t\t\t\t\t\t\t\t\t\t\t\t   string lastBsonId, \r\n\t\t\t\t\t\t\t\t\t\t\t\t   int minPopulation = 0)\r\n{\r\n    Expression&lt;Func&lt;City, bool&gt;&gt; conditions \r\n\t\t\t\t\t\t= (x =&gt; x.CountryCode == countryCode\r\n                               &amp;&amp; x.Population &gt;= minPopulation);\r\n\r\n    ObjectId id;\r\n    if (string.IsNullOrEmpty(lastBsonId) &amp;&amp; ObjectId.TryParse(lastBsonId, out id))\r\n    {\r\n        conditions = (x =&gt; x.CountryCode == countryCode\r\n                        &amp;&amp; x.Population &gt;= minPopulation\r\n                        &amp;&amp; x.Id &lt; id);\r\n    }\r\n\r\n    return conditions;\r\n\r\n}\r\n\r\npublic async Task&lt;object&gt; GetCitiesLinq(string countryCode, \r\n\t\t\t\t\t\t\t\t\t\tstring lastBsonId, \r\n\t\t\t\t\t\t\t\t\t\tint minPopulation = 0)\r\n{\r\n\r\n    try\r\n    {\r\n        var items = await _context.CitiesLinq\r\n                            .Where(GetConditions(countryCode, lastBsonId, minPopulation))\r\n                            .OrderByDescending(x =&gt; x.Id)\r\n                            .Take(200)\r\n                            .ToListAsync();\r\n\r\n        \/\/ select just few elements\r\n        var returnItems = items.Select(x =&gt; new\r\n                            {\r\n                                BsonId = x.Id.ToString(),\r\n                                Timestamp = x.Id.Timestamp,\r\n                                ServerUpdatedOn = x.Id.CreationTime,\r\n                                x.Name,\r\n                                x.CountryCode,\r\n                                x.Population\r\n                            });\r\n\r\n        int countItems = await _context.CitiesLinq\r\n                            .Where(GetConditions(countryCode, &quot;&quot;, minPopulation))\r\n                            .CountAsync();\r\n\r\n\r\n        return new\r\n            {\r\n                count = countItems,\r\n                items = returnItems\r\n            };\r\n    }\r\n    catch (Exception ex)\r\n    {\r\n        \/\/ log or manage the exception\r\n        throw ex;\r\n    }\r\n}\r\n<\/pre>\n<p>and in the controller<\/p>\n<pre>\r\n[NoCache]\r\n[HttpGet]\r\npublic async Task&lt;object&gt; Get(string countryCode, int? population, string lastId)\r\n{\r\n\treturn await _travelItemRepository\r\n\t\t\t\t\t.GetCitiesLinq(countryCode, lastId, population ?? 0);\r\n}\r\n\r\n<\/pre>\n<p>The initial request (sample):<\/p>\n<pre>\r\nhttp:\/\/localhost:61612\/api\/city?countryCode=FR&population=10000\r\n<\/pre>\n<p>followed by other requests where we specify the last retrieved Id:<\/p>\n<pre>\r\nhttp:\/\/localhost:61612\/api\/city?countryCode=FR&population=10000&lastId=58fc8ae631a8a6f8d00101f9\r\n<\/pre>\n<p>Here is just a sample:<br \/>\n<img loading=\"lazy\" src=\"data:image\/svg+xml,%3Csvg xmlns='http:\/\/www.w3.org\/2000\/svg' viewBox='0 0 628 305'%3E%3C\/svg%3E\" data-src=\"https:\/\/qappdesign.com\/code\/wp-content\/uploads\/2017\/08\/mongodb-paging-dotnet-webapi.jpg\" alt=\"\" width=\"628\" height=\"305\" class=\"alignnone size-full wp-image-729 lazyload \" data-sizes=\"auto\" data-srcset=\"https:\/\/i0.wp.com\/qappdesign.com\/code\/wp-content\/uploads\/2017\/08\/mongodb-paging-dotnet-webapi.jpg?w=628&amp;ssl=1 628w, https:\/\/i0.wp.com\/qappdesign.com\/code\/wp-content\/uploads\/2017\/08\/mongodb-paging-dotnet-webapi.jpg?resize=300%2C146&amp;ssl=1 300w\" sizes=\"(max-width: 628px) 100vw, 628px\" \/><noscript><img loading=\"lazy\" src=\"https:\/\/i0.wp.com\/qappdesign.com\/code\/wp-content\/uploads\/2017\/08\/mongodb-paging-dotnet-webapi.jpg?resize=628%2C305&#038;ssl=1\" alt=\"\" width=\"628\" height=\"305\" class=\"alignnone size-full wp-image-729\" srcset=\"https:\/\/i0.wp.com\/qappdesign.com\/code\/wp-content\/uploads\/2017\/08\/mongodb-paging-dotnet-webapi.jpg?w=628&amp;ssl=1 628w, https:\/\/i0.wp.com\/qappdesign.com\/code\/wp-content\/uploads\/2017\/08\/mongodb-paging-dotnet-webapi.jpg?resize=300%2C146&amp;ssl=1 300w\" sizes=\"(max-width: 628px) 100vw, 628px\" data-recalc-dims=\"1\" \/><\/noscript><\/p>\n<h4>At the end<\/h4>\n<p>I hope this helps, and please let me know if you need to be extended or have questions.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>What is the best way (performance wise) to paginate results in MongoDB ? Especially when you also want to get the total number of results ? Project running with .NET Core 2.0 Where to start ? For answering to these questions, let&#8217;s start from the datasets defined in my earlier article Part 1: How to search good places to travel (MongoDb LINQ &#038; .NET Core). That article was quick introduction, on how to load big chunks of data and then retrieve values using WebApi and LINQ. Here, I will start from that project, extending it with more details related to paging the query results. You could also check Part 3 &#8211; MongoDb and LINQ: How to aggregate and join collections You can find the full solution, together with the data here: https:\/\/github.com\/fpetru\/WebApiQueryMongoDb Topics covered Paging query results with skip and limit Paging query results using last position MongoDb BSonId Paging using MongoDb .NET Driver To install Here are all the things needed to be installed: Visual Studio Community 2017, including .NET Core option MongoDB and Robomongo See the results Here are few steps to have the solution ready, and see the results immediately: Clone or download the project Run import.bat [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"spay_email":"","jetpack_publicize_message":"Paging in MongoDB - How to actually avoid poor performance ? #NoSQL #dotNET #MongoDB #LINQ #Csharp","jetpack_is_tweetstorm":false,"jetpack_publicize_feature_enabled":true},"categories":[13,6,7,8],"tags":[22,3],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v18.0 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Paging in MongoDB - How to actually avoid poor performance ? - Cloud, Data and Integrations<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/qappdesign.com\/code\/paging-mongodb-avoid-poor-performance\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Paging in MongoDB - How to actually avoid poor performance ? - Cloud, Data and Integrations\" \/>\n<meta property=\"og:description\" content=\"What is the best way (performance wise) to paginate results in MongoDB ? Especially when you also want to get the total number of results ? Project running with .NET Core 2.0 Where to start ? For answering to these questions, let&#8217;s start from the datasets defined in my earlier article Part 1: How to search good places to travel (MongoDb LINQ &#038; .NET Core). That article was quick introduction, on how to load big chunks of data and then retrieve values using WebApi and LINQ. Here, I will start from that project, extending it with more details related to paging the query results. You could also check Part 3 &#8211; MongoDb and LINQ: How to aggregate and join collections You can find the full solution, together with the data here: https:\/\/github.com\/fpetru\/WebApiQueryMongoDb Topics covered Paging query results with skip and limit Paging query results using last position MongoDb BSonId Paging using MongoDb .NET Driver To install Here are all the things needed to be installed: Visual Studio Community 2017, including .NET Core option MongoDB and Robomongo See the results Here are few steps to have the solution ready, and see the results immediately: Clone or download the project Run import.bat [&hellip;]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/qappdesign.com\/code\/paging-mongodb-avoid-poor-performance\/\" \/>\n<meta property=\"og:site_name\" content=\"Cloud, Data and Integrations\" \/>\n<meta property=\"article:published_time\" content=\"2017-08-16T20:16:34+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2017-11-15T05:44:34+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/qappdesign.com\/code\/wp-content\/uploads\/2017\/06\/mongodb-index-describe.jpg\" \/>\n<meta name=\"twitter:card\" content=\"summary\" \/>\n<meta name=\"twitter:creator\" content=\"@qappdesign\" \/>\n<meta name=\"twitter:site\" content=\"@QAppDesign\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Petru Faurescu\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"7 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebSite\",\"@id\":\"https:\/\/qappdesign.com\/code\/#website\",\"url\":\"https:\/\/qappdesign.com\/code\/\",\"name\":\"QualityAppDesign\",\"description\":\"with Petru Faurescu\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/qappdesign.com\/code\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"ImageObject\",\"@id\":\"https:\/\/qappdesign.com\/code\/paging-mongodb-avoid-poor-performance\/#primaryimage\",\"inLanguage\":\"en-US\",\"url\":\"https:\/\/i0.wp.com\/qappdesign.com\/code\/wp-content\/uploads\/2017\/06\/mongodb-index-describe.jpg?fit=790%2C359&ssl=1\",\"contentUrl\":\"https:\/\/i0.wp.com\/qappdesign.com\/code\/wp-content\/uploads\/2017\/06\/mongodb-index-describe.jpg?fit=790%2C359&ssl=1\",\"width\":790,\"height\":359},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/qappdesign.com\/code\/paging-mongodb-avoid-poor-performance\/#webpage\",\"url\":\"https:\/\/qappdesign.com\/code\/paging-mongodb-avoid-poor-performance\/\",\"name\":\"Paging in MongoDB - How to actually avoid poor performance ? - Cloud, Data and Integrations\",\"isPartOf\":{\"@id\":\"https:\/\/qappdesign.com\/code\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/qappdesign.com\/code\/paging-mongodb-avoid-poor-performance\/#primaryimage\"},\"datePublished\":\"2017-08-16T20:16:34+00:00\",\"dateModified\":\"2017-11-15T05:44:34+00:00\",\"author\":{\"@id\":\"https:\/\/qappdesign.com\/code\/#\/schema\/person\/54db90dc6fe846cfd4c5a9544d93b75a\"},\"breadcrumb\":{\"@id\":\"https:\/\/qappdesign.com\/code\/paging-mongodb-avoid-poor-performance\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/qappdesign.com\/code\/paging-mongodb-avoid-poor-performance\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/qappdesign.com\/code\/paging-mongodb-avoid-poor-performance\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/qappdesign.com\/code\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Paging in MongoDB &#8211; How to actually avoid poor performance ?\"}]},{\"@type\":\"Person\",\"@id\":\"https:\/\/qappdesign.com\/code\/#\/schema\/person\/54db90dc6fe846cfd4c5a9544d93b75a\",\"name\":\"Petru Faurescu\",\"image\":{\"@type\":\"ImageObject\",\"@id\":\"https:\/\/qappdesign.com\/code\/#personlogo\",\"inLanguage\":\"en-US\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/84fb359a4e3d583dbea5a34bd5566956?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/84fb359a4e3d583dbea5a34bd5566956?s=96&d=mm&r=g\",\"caption\":\"Petru Faurescu\"},\"description\":\"Product lead, software developer &amp; architect\",\"sameAs\":[\"https:\/\/qappdesign.com\/code\",\"https:\/\/www.linkedin.com\/in\/petrufaurescu\/\",\"https:\/\/twitter.com\/@qappdesign\"]}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Paging in MongoDB - How to actually avoid poor performance ? - Cloud, Data and Integrations","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/qappdesign.com\/code\/paging-mongodb-avoid-poor-performance\/","og_locale":"en_US","og_type":"article","og_title":"Paging in MongoDB - How to actually avoid poor performance ? - Cloud, Data and Integrations","og_description":"What is the best way (performance wise) to paginate results in MongoDB ? Especially when you also want to get the total number of results ? Project running with .NET Core 2.0 Where to start ? For answering to these questions, let&#8217;s start from the datasets defined in my earlier article Part 1: How to search good places to travel (MongoDb LINQ &#038; .NET Core). That article was quick introduction, on how to load big chunks of data and then retrieve values using WebApi and LINQ. Here, I will start from that project, extending it with more details related to paging the query results. You could also check Part 3 &#8211; MongoDb and LINQ: How to aggregate and join collections You can find the full solution, together with the data here: https:\/\/github.com\/fpetru\/WebApiQueryMongoDb Topics covered Paging query results with skip and limit Paging query results using last position MongoDb BSonId Paging using MongoDb .NET Driver To install Here are all the things needed to be installed: Visual Studio Community 2017, including .NET Core option MongoDB and Robomongo See the results Here are few steps to have the solution ready, and see the results immediately: Clone or download the project Run import.bat [&hellip;]","og_url":"https:\/\/qappdesign.com\/code\/paging-mongodb-avoid-poor-performance\/","og_site_name":"Cloud, Data and Integrations","article_published_time":"2017-08-16T20:16:34+00:00","article_modified_time":"2017-11-15T05:44:34+00:00","og_image":[{"url":"https:\/\/qappdesign.com\/code\/wp-content\/uploads\/2017\/06\/mongodb-index-describe.jpg"}],"twitter_card":"summary","twitter_creator":"@qappdesign","twitter_site":"@QAppDesign","twitter_misc":{"Written by":"Petru Faurescu","Est. reading time":"7 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebSite","@id":"https:\/\/qappdesign.com\/code\/#website","url":"https:\/\/qappdesign.com\/code\/","name":"QualityAppDesign","description":"with Petru Faurescu","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/qappdesign.com\/code\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"ImageObject","@id":"https:\/\/qappdesign.com\/code\/paging-mongodb-avoid-poor-performance\/#primaryimage","inLanguage":"en-US","url":"https:\/\/i0.wp.com\/qappdesign.com\/code\/wp-content\/uploads\/2017\/06\/mongodb-index-describe.jpg?fit=790%2C359&ssl=1","contentUrl":"https:\/\/i0.wp.com\/qappdesign.com\/code\/wp-content\/uploads\/2017\/06\/mongodb-index-describe.jpg?fit=790%2C359&ssl=1","width":790,"height":359},{"@type":"WebPage","@id":"https:\/\/qappdesign.com\/code\/paging-mongodb-avoid-poor-performance\/#webpage","url":"https:\/\/qappdesign.com\/code\/paging-mongodb-avoid-poor-performance\/","name":"Paging in MongoDB - How to actually avoid poor performance ? - Cloud, Data and Integrations","isPartOf":{"@id":"https:\/\/qappdesign.com\/code\/#website"},"primaryImageOfPage":{"@id":"https:\/\/qappdesign.com\/code\/paging-mongodb-avoid-poor-performance\/#primaryimage"},"datePublished":"2017-08-16T20:16:34+00:00","dateModified":"2017-11-15T05:44:34+00:00","author":{"@id":"https:\/\/qappdesign.com\/code\/#\/schema\/person\/54db90dc6fe846cfd4c5a9544d93b75a"},"breadcrumb":{"@id":"https:\/\/qappdesign.com\/code\/paging-mongodb-avoid-poor-performance\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/qappdesign.com\/code\/paging-mongodb-avoid-poor-performance\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/qappdesign.com\/code\/paging-mongodb-avoid-poor-performance\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/qappdesign.com\/code\/"},{"@type":"ListItem","position":2,"name":"Paging in MongoDB &#8211; How to actually avoid poor performance ?"}]},{"@type":"Person","@id":"https:\/\/qappdesign.com\/code\/#\/schema\/person\/54db90dc6fe846cfd4c5a9544d93b75a","name":"Petru Faurescu","image":{"@type":"ImageObject","@id":"https:\/\/qappdesign.com\/code\/#personlogo","inLanguage":"en-US","url":"https:\/\/secure.gravatar.com\/avatar\/84fb359a4e3d583dbea5a34bd5566956?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/84fb359a4e3d583dbea5a34bd5566956?s=96&d=mm&r=g","caption":"Petru Faurescu"},"description":"Product lead, software developer &amp; architect","sameAs":["https:\/\/qappdesign.com\/code","https:\/\/www.linkedin.com\/in\/petrufaurescu\/","https:\/\/twitter.com\/@qappdesign"]}]}},"jetpack_featured_media_url":"","jetpack_publicize_connections":[],"jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p82xvM-aq","_links":{"self":[{"href":"https:\/\/qappdesign.com\/code\/wp-json\/wp\/v2\/posts\/646"}],"collection":[{"href":"https:\/\/qappdesign.com\/code\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/qappdesign.com\/code\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/qappdesign.com\/code\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/qappdesign.com\/code\/wp-json\/wp\/v2\/comments?post=646"}],"version-history":[{"count":65,"href":"https:\/\/qappdesign.com\/code\/wp-json\/wp\/v2\/posts\/646\/revisions"}],"predecessor-version":[{"id":811,"href":"https:\/\/qappdesign.com\/code\/wp-json\/wp\/v2\/posts\/646\/revisions\/811"}],"wp:attachment":[{"href":"https:\/\/qappdesign.com\/code\/wp-json\/wp\/v2\/media?parent=646"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/qappdesign.com\/code\/wp-json\/wp\/v2\/categories?post=646"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/qappdesign.com\/code\/wp-json\/wp\/v2\/tags?post=646"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}