Adding option for multithreaded query mode by GeorgiLH · Pull Request #27 · compiler-research/ramtools

GeorgiLH · 2026-03-07T14:35:38Z

Currently only prints details about the supplied file.

prints file data.

github-actions

clang-tidy made some suggestions

github-actions · 2026-03-07T15:10:13Z

@@ -0,0 +1,9 @@
+#ifndef RAMCORE_RDF_RAMNTUPLEVIEW_H
+#define RAMCORE_RDF_RAMNTUPLEVIEW_H


warning: header guard does not follow preferred style [llvm-header-guard]

Suggested change

#define RAMCORE_RDF_RAMNTUPLEVIEW_H

#ifndef GITHUB_WORKSPACE_INC_RAMCORE_RDF_RAMNTUPLEVIEW_H

#define GITHUB_WORKSPACE_INC_RAMCORE_RDF_RAMNTUPLEVIEW_H

inc/ramcore/RDF_RAMNTupleView.h:7:

- #endif //RAMCORE_RDF_RAMNTUPLEVIEW_H + #endif // GITHUB_WORKSPACE_INC_RAMCORE_RDF_RAMNTUPLEVIEW_H

github-actions · 2026-03-07T15:10:13Z

@@ -0,0 +1,9 @@
+#ifndef RAMCORE_RDF_RAMNTUPLEVIEW_H
+#define RAMCORE_RDF_RAMNTUPLEVIEW_H
+#include <RtypesCore.h>


warning: 'RtypesCore.h' file not found [clang-diagnostic-error]

#include <RtypesCore.h> ^

github-actions · 2026-03-07T15:10:13Z

@@ -0,0 +1,13 @@
+#include <ROOT/RDataFrame.hxx>


warning: 'ROOT/RDataFrame.hxx' file not found [clang-diagnostic-error]

#include <ROOT/RDataFrame.hxx> ^

github-actions · 2026-03-07T15:10:13Z

+#include <ROOT/RDataFrame.hxx>
+#include <ramcore/RDF_RAMNTupleView.h>
+
+ULong64_t rdf_ramntupleview(const char *file, const char *query, bool cache, bool perfstats, const char *perfstatsfilename){


warning: function 'rdf_ramntupleview' can be made static or moved into an anonymous namespace to enforce internal linkage [misc-use-internal-linkage]

Suggested change

ULong64_t rdf_ramntupleview(const char *file, const char *query, bool cache, bool perfstats, const char *perfstatsfilename){

ULong64_t rdf_ramntupleviewstatic (const char *file, const char *query, bool cache, bool perfstats, const char *perfstatsfilename){

github-actions · 2026-03-07T15:10:13Z

+
+
+#include <iostream>
+#include <ramcore/RDF_RAMNTupleview.h>


warning: 'ramcore/RDF_RAMNTupleview.h' file not found [clang-diagnostic-error]

#include <ramcore/RDF_RAMNTupleview.h> ^

github-actions · 2026-03-07T15:10:13Z

+int main(int argc, char **argv){
+
+if (argc < 2){
+      std::cerr << "Usage: " << argv[0] << " <file.root> [rname:start-end]\n";


warning: do not use pointer arithmetic [cppcoreguidelines-pro-bounds-pointer-arithmetic]

std::cerr << "Usage: " << argv[0] << " <file.root> [rname:start-end]\n"; ^

github-actions · 2026-03-07T15:10:13Z

+
+if (argc < 2){
+      std::cerr << "Usage: " << argv[0] << " <file.root> [rname:start-end]\n";
+      std::cerr << "Example: " << argv[0] << " output.root chr1:1000-2000\n";


warning: do not use pointer arithmetic [cppcoreguidelines-pro-bounds-pointer-arithmetic]

std::cerr << "Example: " << argv[0] << " output.root chr1:1000-2000\n"; ^

github-actions · 2026-03-07T15:10:13Z

+      return 1;
+}
+
+const char* file = argv[1];


warning: do not use pointer arithmetic [cppcoreguidelines-pro-bounds-pointer-arithmetic]

const char* file = argv[1]; ^

github-actions · 2026-03-07T15:10:13Z

+
+const char* file = argv[1];
+
+const char* query = argv[2];


warning: do not use pointer arithmetic [cppcoreguidelines-pro-bounds-pointer-arithmetic]

const char* query = argv[2]; ^

AdityaPandeyCN · 2026-03-07T15:16:36Z

@GeorgiLH We can make changes in the existing RAMNTupleview(https://github.com/compiler-research/ramtools/blob/develop/src/ramcore/RAMNTupleView.cxx) tool file with this approach, it would be better to have a single tool for viewing and querying( condition applied that this approach is faster than the previous one). Also make sure that the tests pass locally and also apply clang formatting by running this git-clang-format HEAD~

github-actions

clang-tidy made some suggestions

github-actions · 2026-03-14T08:12:19Z

@@ -0,0 +1,8 @@
+#ifndef RAMCORE_RDF_RAMNTUPLEVIEW_H
+#define RAMCORE_RDF_RAMNTUPLEVIEW_H


warning: header guard does not follow preferred style [llvm-header-guard]

Suggested change

#define RAMCORE_RDF_RAMNTUPLEVIEW_H

#ifndef GITHUB_WORKSPACE_INC_RAMCORE_RDF_RAMNTUPLEVIEW_H

#define GITHUB_WORKSPACE_INC_RAMCORE_RDF_RAMNTUPLEVIEW_H

inc/ramcore/RDF_RAMNTupleView.h:7:

- #endif // RAMCORE_RDF_RAMNTUPLEVIEW_H + #endif // GITHUB_WORKSPACE_INC_RAMCORE_RDF_RAMNTUPLEVIEW_H

github-actions · 2026-03-14T08:12:19Z

+#include <ramcore/RDF_RAMNTupleView.h>
+
+ULong64_t
+rdf_ramntupleview(const char *file, const char *query, bool cache, bool perfstats, const char *perfstatsfilename)


warning: function 'rdf_ramntupleview' can be made static or moved into an anonymous namespace to enforce internal linkage [misc-use-internal-linkage]

Suggested change

rdf_ramntupleview(const char *file, const char *query, bool cache, bool perfstats, const char *perfstatsfilename)

rdf_ramntupleviewstatic (const char *file, const char *query, bool cache, bool perfstats, const char *perfstatsfilename)

github-actions · 2026-03-14T08:12:19Z

@@ -1,4 +1,5 @@
 #include "ramcore/RAMNTupleView.h"


warning: 'ramcore/RAMNTupleView.h' file not found [clang-diagnostic-error]

#include "ramcore/RAMNTupleView.h" ^

codecov-commenter · 2026-03-14T08:28:29Z

Codecov Report

❌ Patch coverage is 0% with 42 lines in your changes missing coverage. Please review.
✅ Project coverage is 63.19%. Comparing base (5cec39c) to head (dc72994).

Files with missing lines	Patch %	Lines
src/ramcore/RAMNTupleView.cxx	0.00%	31 Missing ⚠️
tools/ramntupleview.cxx	0.00%	11 Missing ⚠️

❌ Your patch status has failed because the patch coverage (0.00%) is below the target coverage (85.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files

@@             Coverage Diff             @@
##           develop      #27      +/-   ##
===========================================
- Coverage    64.78%   63.19%   -1.60%     
===========================================
  Files           16       16              
  Lines         1525     1565      +40     
  Branches       632      659      +27     
===========================================
+ Hits           988      989       +1     
- Misses         462      502      +40     
+ Partials        75       74       -1

Flag	Coverage Δ
unittests	`63.19% <0.00%> (-1.60%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
tools/ramntupleview.cxx	`0.00% <0.00%> (ø)`
src/ramcore/RAMNTupleView.cxx	`64.70% <0.00%> (-22.80%)`	⬇️

... and 1 file with indirect coverage changes

Files with missing lines	Coverage Δ
tools/ramntupleview.cxx	`0.00% <0.00%> (ø)`
src/ramcore/RAMNTupleView.cxx	`64.70% <0.00%> (-22.80%)`	⬇️

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

GeorgiLH · 2026-03-25T17:59:12Z

Gives the same results as samtools but uses multiple threads

vgvassilev · 2026-03-30T08:46:01Z

@AdityaPandeyCN, can you take a look?

github-actions

clang-tidy made some suggestions

There were too many comments to post at once. Showing the first 10 out of 26. Check the log or trigger a new build to see more.

github-actions · 2026-03-30T08:51:35Z

+#define RAMCORE_RDF_RAMNTUPLEVIEW_H
+#include <RtypesCore.h>
+
+Long64_t rdf_ramntupleview(const int num_threads, const char *file, const char *query = "", bool cache = true,


warning: parameter 'num_threads' is const-qualified in the function declaration; const-qualification of parameters only has an effect in function definitions [readability-avoid-const-params-in-decls]

Suggested change

Long64_t rdf_ramntupleview(const int num_threads, const char *file, const char *query = "", bool cache = true,

Long64_t rdf_ramntupleview(int num_threads, const char *file, const char *query = "", bool cache = true,

github-actions · 2026-03-30T08:51:35Z

+using namespace ROOT::RDF;
+namespace rdf {
+
+static int GetRefId(ROOT::RDataFrame &df, const std::string &rname)


warning: no header providing "std::string" is directly included [misc-include-cleaner]

src/ramcore/RDF_RAMNTupleView.cxx:2:

- using namespace ROOT::RDF; + #include <string> + using namespace ROOT::RDF;

github-actions · 2026-03-30T08:51:35Z

+   if (rname == "*")
+      return -1;
+
+   auto refs = df.Take<std::vector<std::string>>("rname_refs");


warning: no header providing "std::vector" is directly included [misc-include-cleaner]

src/ramcore/RDF_RAMNTupleView.cxx:2:

- using namespace ROOT::RDF; + #include <vector> + using namespace ROOT::RDF;

github-actions · 2026-03-30T08:51:35Z

+   auto refs = df.Take<std::vector<std::string>>("rname_refs");
+   const auto &vec = refs.GetValue()[0];
+
+   auto it = std::find(vec.begin(), vec.end(), rname);


warning: no header providing "std::find" is directly included [misc-include-cleaner]

src/ramcore/RDF_RAMNTupleView.cxx:1:

- #include <ramcore/RDF_RAMNTupleView.h> + #include <algorithm> + #include <ramcore/RDF_RAMNTupleView.h>

github-actions · 2026-03-30T08:51:35Z

+   const auto &vec = refs.GetValue()[0];
+
+   auto it = std::find(vec.begin(), vec.end(), rname);
+   return (it == vec.end()) ? -1 : std::distance(vec.begin(), it);


warning: narrowing conversion from 'typename iterator_traits<__normal_iterator<const basic_string *, vector<basic_string>>>::difference_type' (aka 'long') to signed type 'int' is implementation-defined [cppcoreguidelines-narrowing-conversions]

return (it == vec.end()) ? -1 : std::distance(vec.begin(), it); ^

github-actions · 2026-03-30T08:51:35Z

+   const auto &vec = refs.GetValue()[0];
+
+   auto it = std::find(vec.begin(), vec.end(), rname);
+   return (it == vec.end()) ? -1 : std::distance(vec.begin(), it);


warning: no header providing "std::distance" is directly included [misc-include-cleaner]

src/ramcore/RDF_RAMNTupleView.cxx:1:

- #include <ramcore/RDF_RAMNTupleView.h> + #include <iterator> + #include <ramcore/RDF_RAMNTupleView.h>

github-actions · 2026-03-30T08:51:35Z

+}
+} // namespace rdf
+
+Long64_t rdf_ramntupleview(const int num_threads, const char *file, const char *query, bool cache, bool perfstats,


warning: no header providing "Long64_t" is directly included [misc-include-cleaner]

src/ramcore/RDF_RAMNTupleView.cxx:1:

- #include <ramcore/RDF_RAMNTupleView.h> + #include <RtypesCore.h> + #include <ramcore/RDF_RAMNTupleView.h>

github-actions · 2026-03-30T08:51:35Z

+Long64_t rdf_ramntupleview(const int num_threads, const char *file, const char *query, bool cache, bool perfstats,
+                           const char *perfstatsfilename)
+{
+   TStopwatch ts;


warning: no header providing "TStopwatch" is directly included [misc-include-cleaner]

src/ramcore/RDF_RAMNTupleView.cxx:0:

- #include <ROOT/RDataFrame.hxx> + #include <TStopwatch.h> + #include <ROOT/RDataFrame.hxx>

github-actions · 2026-03-30T08:51:35Z

+   TStopwatch ts;
+   ts.Start();
+   std::string region = query;
+   int chrDelimiterPos = region.find(":");


warning: 'find' called with a string literal consisting of a single character; consider using the more effective overload accepting a character [performance-faster-string-find]

Suggested change

int chrDelimiterPos = region.find(":");

int chrDelimiterPos = region.find(':');

github-actions · 2026-03-30T08:51:35Z

+   TStopwatch ts;
+   ts.Start();
+   std::string region = query;
+   int chrDelimiterPos = region.find(":");


warning: narrowing conversion from 'size_type' (aka 'unsigned long') to signed type 'int' is implementation-defined [cppcoreguidelines-narrowing-conversions]

int chrDelimiterPos = region.find(":"); ^

AdityaPandeyCN · 2026-04-06T08:35:53Z

@GeorgiLH We can make changes in the existing RAMNTupleview(https://github.com/compiler-research/ramtools/blob/develop/src/ramcore/RAMNTupleView.cxx) tool file with this approach, it would be better to have a single tool for viewing and querying( condition

We should have a single file for viewing

GeorgiLH · 2026-04-06T14:58:53Z

@GeorgiLH We can make changes in the existing RAMNTupleview(https://github.com/compiler-research/ramtools/blob/develop/src/ramcore/RAMNTupleView.cxx) tool file with this approach, it would be better to have a single tool for viewing and querying( condition

We should have a single file for viewing

Forgot about that. All done now

GeorgiLH · 2026-04-11T15:29:57Z

@AdityaPandeyCN, @vgvassilev i made some changes; could you check them and run the workflows

github-actions

clang-tidy made some suggestions

github-actions · 2026-04-12T13:57:16Z

 Long64_t ramntupleview(const char *file, const char *query = "", bool cache = true, bool perfstats = false,
                       const char *perfstatsfilename = "perf.root");
-
+ULong64_t mt_ramntupleview(int numthreads, const char *file, const char *query = "", bool cache = true, bool perfstats = false,


warning: no header providing "ULong64_t" is directly included [misc-include-cleaner]

ULong64_t mt_ramntupleview(int numthreads, const char *file, const char *query = "", bool cache = true, bool perfstats = false, ^

github-actions · 2026-04-12T13:57:16Z

+   const auto &refids = refs.GetValue()[0];
+
+   auto it = std::find(refids.begin(), refids.end(), rname);
+   return (it == refids.end()) ? -1 : std::distance(refids.begin(), it);


warning: narrowing conversion from 'typename iterator_traits<__normal_iterator<const basic_string *, vector<basic_string>>>::difference_type' (aka 'long') to signed type 'int' is implementation-defined [cppcoreguidelines-narrowing-conversions]

return (it == refids.end()) ? -1 : std::distance(refids.begin(), it); ^

github-actions · 2026-04-12T13:57:16Z

+   const auto &refids = refs.GetValue()[0];
+
+   auto it = std::find(refids.begin(), refids.end(), rname);
+   return (it == refids.end()) ? -1 : std::distance(refids.begin(), it);


warning: no header providing "std::distance" is directly included [misc-include-cleaner]

src/ramcore/RAMNTupleView.cxx:9:

- #include <string> + #include <iterator> + #include <string>

github-actions · 2026-04-12T13:57:16Z

-}
+}
+//NOLINTNEXTLINE(misc-use-internal-linkage)
+ULong64_t mt_ramntupleview(const int numthreads, const char *file, const char *query, bool /*cache*/, bool /*perfstats*/,


warning: no header providing "ULong64_t" is directly included [misc-include-cleaner]

ge) ^

github-actions · 2026-04-12T13:57:16Z

+   bool mt_set = false;
+   int num_threads = 0;
+   if (argc > 3) {
+      std::string mt_flag = argv[3];


warning: do not use pointer arithmetic [cppcoreguidelines-pro-bounds-pointer-arithmetic]

std::string mt_flag = argv[3]; ^

github-actions · 2026-04-12T13:57:17Z

+   bool mt_set = false;
+   int num_threads = 0;
+   if (argc > 3) {
+      std::string mt_flag = argv[3];


warning: no header providing "std::string" is directly included [misc-include-cleaner]

tools/ramntupleview.cxx:4:

- int main(int argc, char *argv[]) + #include <string> + int main(int argc, char *argv[])

github-actions · 2026-04-12T13:57:17Z

+      std::string mt_flag = argv[3];
+      if (mt_flag.find("-m") != std::string::npos) {
+         mt_set = true;
+         num_threads = std::atoi(mt_flag.substr(2).data());


warning: no header providing "std::atoi" is directly included [misc-include-cleaner]

tools/ramntupleview.cxx:1:

- #include <iostream> + #include <cstdlib> + #include <iostream>

github-actions · 2026-04-12T13:57:17Z

-   Long64_t read_count = ramntupleview(file, region_str);
-
-   printf("Found %lld records in region %s\n", read_count, region_str);
+   ULong64_t read_count;


warning: no header providing "ULong64_t" is directly included [misc-include-cleaner]

tools/ramntupleview.cxx:1:

- #include <iostream> + #include <RtypesCore.h> + #include <iostream>

github-actions · 2026-04-12T13:57:17Z

-   Long64_t read_count = ramntupleview(file, region_str);
-
-   printf("Found %lld records in region %s\n", read_count, region_str);
+   ULong64_t read_count;


warning: variable 'read_count' is not initialized [cppcoreguidelines-init-variables]

Suggested change

ULong64_t read_count;

ULong64_t read_count = 0;

github-actions · 2026-04-12T13:57:17Z

+   } else {
+      read_count = ramntupleview(file, region_str);
+   }
+   printf("Found %lld records in region %s", read_count, region_str);


warning: do not call c-style vararg functions [cppcoreguidelines-pro-type-vararg]

printf("Found %lld records in region %s", read_count, region_str); ^

GeorgiLH · 2026-04-14T15:24:15Z

Squashed last two commits and applied clang-format

AdityaPandeyCN · 2026-04-14T16:12:01Z

Can you attach a result showing the speedups?

GeorgiLH · 2026-04-15T10:55:15Z

This version does not show speedups because it does not have indexing. I wanted to get this merged before making pull requests about flag filtering and indexing although I could just push the commits to this PR.

The beginning of the RDataFrame query tool for ramtools. Currently only

f953702

prints file data.

github-actions bot reviewed Mar 7, 2026

View reviewed changes

Fix linking errors and remove unnecessary file

d18ecdd

github-actions bot reviewed Mar 14, 2026

View reviewed changes

GeorgiLH added 3 commits March 17, 2026 20:19

Added functionality for querying without indexes

a6c7d28

Add basic functionality for querying

aa5df5b

Merge branch 'develop' into rdataframe

aed685f

GeorgiLH marked this pull request as ready for review March 25, 2026 17:58

GeorgiLH changed the title ~~The RDataFrame query tool made for concurrent io~~ Adding option for multithreaded query mode Mar 25, 2026

github-actions bot reviewed Mar 30, 2026

View reviewed changes

github-actions bot reviewed Apr 12, 2026

View reviewed changes

Move changes into existing file

10d62d8

GeorgiLH force-pushed the rdataframe branch from dc72994 to 10d62d8 Compare April 14, 2026 15:21

		@@ -0,0 +1,9 @@
		#ifndef RAMCORE_RDF_RAMNTUPLEVIEW_H
		#define RAMCORE_RDF_RAMNTUPLEVIEW_H

	#define RAMCORE_RDF_RAMNTUPLEVIEW_H
	#ifndef GITHUB_WORKSPACE_INC_RAMCORE_RDF_RAMNTUPLEVIEW_H
	#define GITHUB_WORKSPACE_INC_RAMCORE_RDF_RAMNTUPLEVIEW_H

	ULong64_t rdf_ramntupleview(const char file, const char query, bool cache, bool perfstats, const char *perfstatsfilename){
	ULong64_t rdf_ramntupleviewstatic (const char file, const char query, bool cache, bool perfstats, const char *perfstatsfilename){

		@@ -0,0 +1,8 @@
		#ifndef RAMCORE_RDF_RAMNTUPLEVIEW_H
		#define RAMCORE_RDF_RAMNTUPLEVIEW_H

	rdf_ramntupleview(const char file, const char query, bool cache, bool perfstats, const char *perfstatsfilename)
	rdf_ramntupleviewstatic (const char file, const char query, bool cache, bool perfstats, const char *perfstatsfilename)

	Long64_t rdf_ramntupleview(const int num_threads, const char file, const char query = "", bool cache = true,
	Long64_t rdf_ramntupleview(int num_threads, const char file, const char query = "", bool cache = true,

	int chrDelimiterPos = region.find(":");
	int chrDelimiterPos = region.find(':');

Conversation

GeorgiLH commented Mar 7, 2026

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot Mar 7, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions bot Mar 7, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions bot Mar 7, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions bot Mar 7, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions bot Mar 7, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions bot Mar 7, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions bot Mar 7, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions bot Mar 7, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions bot Mar 7, 2026

Choose a reason for hiding this comment

Uh oh!

AdityaPandeyCN commented Mar 7, 2026

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot Mar 14, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions bot Mar 14, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions bot Mar 14, 2026

Choose a reason for hiding this comment

Uh oh!

codecov-commenter commented Mar 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

GeorgiLH commented Mar 25, 2026

Uh oh!

vgvassilev commented Mar 30, 2026

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot Mar 30, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions bot Mar 30, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions bot Mar 30, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions bot Mar 30, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions bot Mar 30, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions bot Mar 30, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions bot Mar 30, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions bot Mar 30, 2026

codecov-commenter commented Mar 14, 2026 •

edited

Loading