Skip to content

Commit 143cb35

Browse files
Merge pull request #332 from pitrou/fundable-arrow-ree
Add fundable project for Arrow run-end-encoding
2 parents 3abb847 + ac75274 commit 143cb35

4 files changed

Lines changed: 81 additions & 0 deletions

File tree

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
#### Overview
2+
3+
Apache Arrow is the universal columnar format and multi-language toolbox for fast data interchange and in-memory analytics.
4+
5+
Run-End-Encoding is a composite Arrow datatype that efficiently represents data with many repeated values, such as timeseries.
6+
While basic support is present, Run-End-Encoding is not universally supported in all Arrow C++ components.
7+
8+
We propose to finish implementing support for Run-End-Encoding in all components of Arrow C++:
9+
10+
* cast kernels (including implicit run-end-encoding or decoding where applicable)
11+
12+
* scalar kernels:
13+
- all unary kernels (math, etc.)
14+
- all binary kernels (arithmetic, comparisons, etc.)
15+
- `coalesce`, `if_else`, `case_when`
16+
- fix incorrect output in `is_null` and `true_unless_null`
17+
18+
* vector kernels:
19+
- cumulative aggregations (`cumulative_max`, `cumulative_sum`, etc.)
20+
- `pairwise_diff`
21+
- `filter`, `take`, `scatter`
22+
- `fill_null_forward`, `fill_null_backward`
23+
- `replace_with_mask`
24+
- `indices_non_zero`
25+
- `sort_indices`, `rank`, `rank_normal`, `rank_quantile`
26+
- `partition_nth_indices`
27+
- `select_k_unstable`
28+
- `unique`, `value_counts`
29+
- `winsorize`
30+
- fix incorrect output in `drop_null`
31+
32+
* aggregate kernels:
33+
- `all`, `any`
34+
- `count_distinct`
35+
- `approximate_median`
36+
- `sum`, `product`, `mean`, `mode`, `quantile`, `tdigest`
37+
- `variance`, `stddev`, `skew`, `kurtosis`
38+
- `first`, `last`, `min`, `max`
39+
- `index`
40+
- fix incorrect output in `count`
41+
42+
* Parquet reader and writer
43+
44+
* CSV reader
45+
46+
* ORC reader and writer
47+
48+
Funders can decide to fund the entire package, or choose the components they are interested in.
49+
50+
##### Are you interested in this project? Either entirely or partially, contact us for more information on how to help us fund it.

src/components/fundable/projectsDetails.ts

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@ import MatrixOperationsInXtensorMD from "@site/src/components/fundable/descripti
88
import BinaryViewInArrowCppMD from "@site/src/components/fundable/descriptions/BinaryViewInArrowCpp.md"
99
import Decimal32InArrowCppMD from "@site/src/components/fundable/descriptions/Decimal32InArrowCpp.md"
1010
import Float16InArrowCppMD from "@site/src/components/fundable/descriptions/Float16InArrowCpp.md"
11+
import RunEndEncodedInArrowCppMD from "@site/src/components/fundable/descriptions/RunEndEncodedInArrowCpp.md"
1112
import ParquetNullOptimizationsMD from "@site/src/components/fundable/descriptions/ParquetNullOptimizations.md"
1213

1314
export const fundableProjectsDetails = {
@@ -140,6 +141,18 @@ export const fundableProjectsDetails = {
140141
currentFundingPercentage: 0,
141142
repoLink: "https://github.com/apache/arrow"
142143
},
144+
{
145+
category: "Apache Arrow and Parquet",
146+
title: "Complete Run-End-Encoded support in Arrow C++",
147+
pageName: "RunEndEncodedInApacheArrow",
148+
shortDescription: "Like dictionary encoding, run-end-encoding allows representing some kinds of data more efficiently.",
149+
description: RunEndEncodedInArrowCppMD,
150+
price: "TBD",
151+
maxNbOfFunders: 4,
152+
currentNbOfFunders: 0,
153+
currentFundingPercentage: 0,
154+
repoLink: "https://github.com/apache/arrow"
155+
},
143156
{
144157
category: "Apache Arrow and Parquet",
145158
title: "Parquet reader optimizations",
Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
import useDocusaurusContext from '@docusaurus/useDocusaurusContext';
2+
import GetAQuotePage from '@site/src/components/fundable/GetAQuotePage';
3+
4+
export default function FundablePage() {
5+
const { siteConfig } = useDocusaurusContext();
6+
return (
7+
<GetAQuotePage/>
8+
);
9+
}
Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
import useDocusaurusContext from '@docusaurus/useDocusaurusContext';
2+
import LargeProjectCardPage from '@site/src/components/fundable/LargeProjectCardPage';
3+
4+
export default function FundablePage() {
5+
const { siteConfig } = useDocusaurusContext();
6+
return (
7+
<LargeProjectCardPage/>
8+
);
9+
}

0 commit comments

Comments
 (0)