I am using Startup.cs for my configuration.
app.UseRotativa() in Configure(IApplicationBuilder app, IWebHostEnvironment env, ILoggerFactory loggerFactory) method gives me a nice little error that Rotativa can only be used in a WebApplication :/
Well, turns out I can simply do RotativaConfiguration.Setup(env.WebRootPath); in Startup.cs, bypassing the extension method completely.