Allow combine_below on each layer

README.md

@@ -128,6 +128,12 @@ Bug reports, suggestions and (especially!) pull requests are very welcome on the
 
 Formatting: braces and indents as shown, hard tabs (4sp). (Yes, I know.) Please be conservative about adding dependencies or increasing the memory requirement.
 
+To safely rebuild the project after making changes to the code, you need to run :
+
+    make clean
+    make
+    sudo make install
+
 ## Copyright
 
 tilemaker is maintained by Richard Fairhurst and supported by [many contributors](https://github.com/systemed/tilemaker/graphs/contributors). We particularly celebrate the invaluable contributions of Wouter van Kleunen, who passed away in 2022.

docs/CONFIGURATION.md

@@ -33,7 +33,7 @@ It also includes these global settings:
 * `basezoom` - the zoom level for which tilemaker will generate tiles internally (should usually be the same as `maxzoom`)
 * `include_ids` - whether you want to store the OpenStreetMap IDs for each way/node within your vector tiles. This option is not compatible with the merging options defined by the `combine_xxx` settings (see the dedicated paragraph below)
 * `compress` - for mbtiles output, whether to compress vector tiles (Any of "gzip","deflate" or "none"(default)). For pmtiles output, compression is hardcoded to gzip
-* `combine_below` - whether to merge adjacent linestrings of the same type: will be done at zoom levels below that specified here (e.g. `"combine_below": 14` to merge at z1-13)
+* `combine_below` - whether to merge all linestrings in the tile with the same attributes: will be done at zoom levels below that specified here (e.g. `"combine_below": 14` to merge at z1-13). This global setting will be overridden by the layer-specific parameter `combine_lines_below` in layers where it has been defined (see below).
 * `name`, `version` and `description` - about your project (these are written into the MBTiles file)
 * `high_resolution` (optional) - whether to use extra coordinate precision at the maximum zoom level (makes tiles a bit bigger)
 * `bounding_box` (optional) - the bounding box to output, in [minlon, minlat, maxlon, maxlat] order
@@ -83,6 +83,7 @@ You can add optional parameters to layers:
 * `feature_limit` - restrict the number of features written to each tile
 * `feature_limit_below` - restrict only below this zoom level
 * `combine_polygons_below` - merge adjacent polygons with the same attributes below this zoom level
+* `combine_lines_below` - whether to merge all linestrings in the tile with the same attributes. If not defined, the global setting `combine_below` will be used.
 * `combine_points` - merge points with the same attributes (defaults to `true`: specify `false` to disable)
 * `z_order_ascending` - sort features in ascending order by a numeric value set in the Lua processing script (defaults to `true`: specify `false` for descending order)
 
@@ -121,9 +122,9 @@ For example:
 Be careful when using both the `include_ids: true` setting to include IDs and the `combine_xxx` settings to lighten tiles : they are not compatible. During the merging process, items with identical tags are combined in a collection, with only 1 ID being retained for all the merged items. If you need to have the exact ID for each item (for example, for a clickable map), you need to remove the merging settings in the target levels and layers:
 
 * set `combine_points: false` (`true` is the default value) in the target layers
-* set `combine_below` and `combine_polygons_below` below the target zoom level (or remove them)
+* set `combine_below` in global settings, or `combine_lines_below` and `combine_polygons_below` in layer properties, below the target zoom level (or remove them)
 
-If you need the include OSM types as well, you will need to modify the `process.lua` script, check issue [#740](https://github.com/systemed/tilemaker/issues/740) for more information.
+If you need the include OSM types as well, you can use the `OsmType()`function in your `process.lua` script.
 
 ## Lua processing reference
 
@@ -162,7 +163,7 @@ To do that, you use these methods:
 * `Id()`: get the OSM ID of the current object.
 * `IsClosed()`: returns true if the current object is a closed area.
 * `IsMultiPolygon()`: returns true if the current object is a multipolygon.
-* `ZOrder(number)`: Set a numeric value (default 0) used to sort features within a layer. Use this feature to ensure a proper rendering order if the rendering engine itself does not support sorting. Sorting is not supported across layers merged with `write_to`. Features with different z-order are not merged if `combine_below` or `combine_polygons_below` is used. Use this in conjunction with `feature_limit` to only write the most important (highest z-order) features within a tile. (Values can be -50,000,000 to 50,000,000 and are lossy, particularly beyond -1000 to 1000.)
+* `ZOrder(number)`: Set a numeric value (default 0) used to sort features within a layer. Use this feature to ensure a proper rendering order if the rendering engine itself does not support sorting. Sorting is not supported across layers merged with `write_to`. Features with different z-order are not merged if `combine_below`, `combine_lines_below` or `combine_polygons_below` is used. Use this in conjunction with `feature_limit` to only write the most important (highest z-order) features within a tile. (Values can be -50,000,000 to 50,000,000 and are lossy, particularly beyond -1000 to 1000.)
 * `MinZoom(zoom)`: set the minimum zoom level (0-15) at which this object will be written. Note that the JSON layer configuration minimum still applies (so `:MinZoom(5)` will have no effect if your layer only starts at z6).
 * `Length()` and `Area()`: return the length (metres)/area (square metres) of the current object. Requires Boost 1.67+.
 * `Centroid()`: return the lat/lon of the centre of the current object as a two-element Lua table (element 1 is lat, 2 is lon).

include/shared_data.h

@@ -26,11 +26,12 @@ struct LayerDef {
 	uint simplifyAlgo;
 	uint filterBelow;
 	double filterArea;
-	uint combinePolygonsBelow;
 	bool sortZOrderAscending;
 	uint featureLimit;
 	uint featureLimitBelow;
 	bool combinePoints;
+	uint combineLinesBelow;
+	uint combinePolygonsBelow;
 	std::string source;
 	std::vector<std::string> sourceColumns;
 	bool allSourceColumns;
@@ -58,8 +59,8 @@ class LayerDefinition {
 	// Define a layer (as read from the .json file)
 	uint addLayer(std::string name, uint minzoom, uint maxzoom,
 			uint simplifyBelow, double simplifyLevel, double simplifyLength, double simplifyRatio, uint simplifyAlgo,
-			uint filterBelow, double filterArea, uint combinePolygonsBelow, bool sortZOrderAscending,
-			uint featureLimit, uint featureLimitBelow, bool combinePoints,
+			uint filterBelow, double filterArea, bool sortZOrderAscending,
+			uint featureLimit, uint featureLimitBelow, bool combinePoints, uint combineLinesBelow, uint combinePolygonsBelow,
 			const std::string &source,
 			const std::vector<std::string> &sourceColumns,
 			bool allSourceColumns,

src/shared_data.cpp

@@ -139,8 +139,8 @@ void SharedData::writePMTilesBounds() {
 // Define a layer (as read from the .json file)
 uint LayerDefinition::addLayer(string name, uint minzoom, uint maxzoom,
 		uint simplifyBelow, double simplifyLevel, double simplifyLength, double simplifyRatio, uint simplifyAlgo,
-		uint filterBelow, double filterArea, uint combinePolygonsBelow, bool sortZOrderAscending,
-		uint featureLimit, uint featureLimitBelow, bool combinePoints,
+		uint filterBelow, double filterArea, bool sortZOrderAscending,
+		uint featureLimit, uint featureLimitBelow, bool combinePoints, uint combineLinesBelow, uint combinePolygonsBelow,
 		const std::string &source,
 		const std::vector<std::string> &sourceColumns,
 		bool allSourceColumns,
@@ -150,7 +150,7 @@ uint LayerDefinition::addLayer(string name, uint minzoom, uint maxzoom,
 
 	bool isWriteTo = !writeTo.empty();
 	LayerDef layer = { name, minzoom, maxzoom, simplifyBelow, simplifyLevel, simplifyLength, simplifyRatio, simplifyAlgo,
-		filterBelow, filterArea, combinePolygonsBelow, sortZOrderAscending, featureLimit, featureLimitBelow, combinePoints,
+		filterBelow, filterArea, sortZOrderAscending, featureLimit, featureLimitBelow, combinePoints, combineLinesBelow, combinePolygonsBelow,
 		source, sourceColumns, allSourceColumns, indexed, indexName,
 		std::map<std::string,uint>(), isWriteTo };
 	layers.push_back(layer);
@@ -317,10 +317,11 @@ void Config::readConfig(rapidjson::Document &jsonConfig, bool &hasClippingBox, B
 		double simplifyRatio  = it->value.HasMember("simplify_ratio" ) ? it->value["simplify_ratio" ].GetDouble() : 2.0;
 		int    filterBelow    = it->value.HasMember("filter_below"   ) ? it->value["filter_below"   ].GetInt()    : 0;
 		double filterArea     = it->value.HasMember("filter_area"    ) ? it->value["filter_area"    ].GetDouble() : 0.5;
-		int    combinePolyBelow=it->value.HasMember("combine_polygons_below") ? it->value["combine_polygons_below"].GetInt() : 0;
 		int    featureLimit   = it->value.HasMember("feature_limit"  ) ? it->value["feature_limit"  ].GetInt()    : 0;
 		int  featureLimitBelow= it->value.HasMember("feature_limit_below") ? it->value["feature_limit_below"].GetInt() : (maxZoom+1);
 		bool combinePoints    = it->value.HasMember("combine_points" ) ? it->value["combine_points" ].GetBool()   : true;
+		int combineLinesBelow = it->value.HasMember("combine_lines_below"  ) ? it->value["combine_lines_below"  ].GetInt()    : combineBelow;
+		int    combinePolyBelow=it->value.HasMember("combine_polygons_below") ? it->value["combine_polygons_below"].GetInt() : 0;
 		bool sortZOrderAscending = it->value.HasMember("z_order_ascending") ? it->value["z_order_ascending"].GetBool() : (featureLimit==0);
 		string algo           = it->value.HasMember("simplify_algorithm") ? it->value["simplify_algorithm"].GetString() : "";
 		uint simplifyAlgo = algo=="visvalingam" ? LayerDef::VISVALINGAM : LayerDef::DOUGLAS_PEUCKER;
@@ -343,11 +344,14 @@ void Config::readConfig(rapidjson::Document &jsonConfig, bool &hasClippingBox, B
 
 		layers.addLayer(layerName, minZoom, maxZoom,
 				simplifyBelow, simplifyLevel, simplifyLength, simplifyRatio, simplifyAlgo,
-				filterBelow, filterArea, combinePolyBelow, sortZOrderAscending, featureLimit, featureLimitBelow, combinePoints,
+				filterBelow, filterArea, sortZOrderAscending, featureLimit, featureLimitBelow, combinePoints, combineLinesBelow, combinePolyBelow,
 				source, sourceColumns, allSourceColumns, indexed, indexName,
 				writeTo);
 
 		cout << "Layer " << layerName << " (z" << minZoom << "-" << maxZoom << ")";
+		cout << " - combine points: " << combinePoints;
+		cout << " - combine lines below " << combineLinesBelow;
+		cout << " - combine polygons below " << combinePolyBelow;
 		if (it->value.HasMember("write_to")) { cout << " -> " << it->value["write_to"].GetString(); }
 		cout << endl;
 	}

src/tile_worker.cpp

@@ -277,13 +277,13 @@ void ProcessObjects(
 	double simplifyLevel,
 	unsigned simplifyAlgo,
 	double filterArea,
-	bool combinePolygons,
 	bool combinePoints,
+	bool combineLines,
+	bool combinePolygons,
 	unsigned zoom,
 	const TileBbox &bbox,
 	vtzero::layer_builder& vtLayer
 ) {
-
 	for (auto jt = ooSameLayerBegin; jt != ooSameLayerEnd; ++jt) {
 		OutputObjectID oo = *jt;
 		if (zoom < oo.oo.minZoom) { continue; }
@@ -330,7 +330,7 @@ void ProcessObjects(
 			}
 
 			//This may increment the jt iterator
-			if (oo.oo.geomType == LINESTRING_ && zoom < sharedData.config.combineBelow) {
+			if (oo.oo.geomType == LINESTRING_ && combineLines) {
 				// Append successive linestrings, then reorder afterwards
 				while (jt<(ooSameLayerEnd-1) && oo.oo.compatible((jt+1)->oo)) {
 					jt++;
@@ -440,7 +440,6 @@ void ProcessLayer(
 		if (zoom < ld.filterBelow) { 
 			filterArea = meter2degp(ld.filterArea, latp) * pow(2.0, (ld.filterBelow-1) - zoom);
 		}
-
 		for (size_t i=0; i<sources.size(); i++) {
 			// Loop through output objects
 			auto ooListSameLayer = getObjectsAtSubLayer(data[i], layerNum);
@@ -449,7 +448,7 @@ void ProcessLayer(
 			ProcessObjects(sources[i], attributeStore, 
 				ooListSameLayer.first, end, sharedData, 
 				simplifyLevel, ld.simplifyAlgo,
-				filterArea, zoom < ld.combinePolygonsBelow, ld.combinePoints, zoom, bbox, vtLayer);
+				filterArea, ld.combinePoints, zoom < ld.combineLinesBelow, zoom < ld.combinePolygonsBelow, zoom, bbox, vtLayer);
 		}
 	}
 	if (verbose && std::time(0)-start>3) {