Grab the first 5000 messages

[mmendoza-apa]

I need help trying to perform operations when pulling data. I want take 1000 messages when pulling the data. The only thing I have been able to get working is doing configure.Query.Where(m => m.CreatedAt >= DateTime.Today.AddDays(-7)). Is there any way to Take when doing a pull? Im sure I am missing some sort of connection.

var pullResult2 = await _context.PullAsync(builder =>
        {
            builder.SetParallelOperations(2);
            builder.AddPullRequest<InboxMessage1>(configure =>
            {
                configure.Query.Queryable.Take(5000);
            });
            builder.SaveAfterEveryServiceRequest(false);
        });

[monkeyK1n9]

Hey @mmendoza-apa try to follow these steps and let me know:

1- You have to make sure that your pull size on the server even permits this much. You can configure it in your controller.

Something like this:

[Route("tables/[controller]")]
public class MoviesController : TableController<Movie>
{
  public MoviesController(AppDbContext context) : base()
  {
    Repository = new EntityTableRepository<Movie>(context);
    Options = new TableControllerOptions { PageSize = 10_000 }; // <- HERE
  }
}

2- Datasync supports OData query parameters, so on the client side, you will specify the query parameters in the queryparameters property.

PullResult pullResult = await dbContext.PullAsync(cfg =>
{
    cfg.AddPullRequest<Movie>(options =>
    {
        options.Endpoint = new Uri("/tables/movies", UriKind.Relative);
        options.Query.QueryParameters.Add("$skip", "1000"); // skip the first 1000
        options.Query.QueryParameters.Add("$top", "5000"); // take the next 5000
    });
});

In the example above, we skip 1000 and take the next 5000 as you wanted. There are other OData supported query paramaters you can apply, as from the docs

$filter specifies an OData filter, which is then translated into a LINQ query before being sent to the service.
$select specifies a list of properties that you want to return.
$orderby specifies the properties that you want to use for server-side sorting (including asc for ascending and desc for descending).
$skip tells the server to skip the first N matching entities.
$top tells the server to include the first N matching entities after the skip value.
__includedeleted=true tells the server to include entities that are marked for deletion.

[mmendoza-apa]

Ahh, thank you. I will give it a try and let you know how it works. Currently, we are pulling about 30k objects which takes ~4-5 minutes with parallel options set to 2.

[mmendoza-apa]

@monkeyK1n9 , thank you for your help!

[mmendoza-apa]

I will mention that I do notifice that

PullResult pullResult = await dbContext.PullAsync(cfg =>
{
    cfg.AddPullRequest<Movie>(options =>
    {
        options.Endpoint = new Uri("/tables/movies", UriKind.Relative);
        options.Query.QueryParameters.Add("$top", "500"); // take the next 5000
        options.Query.QueryParameters.Add("$orderby", "createdAt");
    });
});

Whenever I add the orderby, createdAt. I still see orderby=updateAt in my call.

https://localhost:7254/api/v1/inboxmessage?$orderby=updatedAt&$count=true&__includedeleted=true&%24orderby=createdAt&%24top=500