[PATCH v1] Merge ODP linux-generic v1.50.0.0

.github/workflows/coverity.yml

@@ -4,10 +4,10 @@
   schedule:
     - cron: '0 0 * * *' # Once every day at 00:00 UTC
 env:
   CC: gcc
   ARCH: x86_64
   CONTAINER_NAMESPACE: ghcr.io/opendataplane/odp-docker-images
-  OS: ubuntu_20.04
+  OS: ubuntu_24.04
   COVERITY_EMAIL: dummy@opendataplane.org
   COVERITY_PROJECT: ODP-DPDK
 

CHANGELOG

@@ -1,5 +1,128 @@
 # Changelog
 
+## OpenDataPlane (1.50.0.0)
+
+### Backward incompatible API changes
+
+#### IPsec
+* Change `odp_ipsec_out_inline()` function specification to state that packets
+that share data with other packets may not be sent using this function unless
+the pktout to be used has the relevant capability.
+
+#### Packet
+* Redefine the semantics of `odp_packet_ref()` in a way that makes it simpler to
+support packet data sharing. Also, specify better rules for packet data layout
+manipulation operations, allowing private packet data also in the tail of
+packets that reference other packets.
+* Change `odp_packet_ref_pkt()` function specification to match updated
+`odp_packet_ref()` specification.
+* Change `odp_packet_has_ref()` to indicate whether a packet is referenced by
+another packet, not whether the packet shares data with another packet. In
+particular, `odp_packet_ref()` returns a packet that shares data with the given
+packet but is not itself referenced by other packets.
+
+#### Packet IO
+* Add `odp_pktio_capability_t.packet_ref` capability to indicate whether a pktio
+supports sending packets with shared data without using the `dont_free` flag.
+
+#### Timer
+* Change periodic timer expiration event configuration. Instead of configuring a
+single event to be sent for periodic timer expirations, application now
+configures a timeout pool to be created as part of timer pool creation
+(`odp_timer_pool_param_t.periodic`).
+* Add `odp_timer_periodic_param_init()` function for initializing periodic timer
+parameters (`odp_timer_periodic_param_t`).
+* Add `odp_timer_periodic_alloc()` function for allocating periodic timers. Old
+`odp_timer_alloc()` function is used only for single shot timers.
+* Change `odp_timer_periodic_start_t` to only include first expiration tick. The
+remaining previous parameters have been moved to `odp_timer_periodic_param_t`.
+* Change `odp_timer_periodic_ack()` return values to not have a specific value
+for events from a cancelled timer (not last).
+* Add `ODP_TIMER_BUSY` return value to `odp_timer_start()`,
+`odp_timer_restart()`, and `odp_timer_periodic_start()` functions.
+* Add alternative method for configuring periodic timer frequency for timer
+pools and timers. The new method introduces a new periodic timer type
+`ODP_TIMER_TYPE_PERIODIC_FREQ` to be used with periodic timer capability checks
+and timer pool creation. Old periodic timer type `ODP_TIMER_TYPE_PERIODIC` is
+renamed to `ODP_TIMER_TYPE_PERIODIC_BASE_MUL` to better reflect the differences
+between these two types.
+
+#### Traffic Manager
+* Add `odp_tm_capabilities_t.packet_ref` capability to indicate whether packets
+that reference other packets or are referenced by other packets can be enqueued
+to TM.
+
+### Backward compatible API changes
+
+#### Event Vector
+* Add `odp_event_aggr_capability_t.stats` capability for supported statistics
+counters.
+
+#### Packet
+* Add `odp_packet_is_referencing()` function for testing whether a packet
+references another packet but is not a static reference.
+* Clarify how `odp_packet_free()` function operates with packets that
+participate in packet data sharing.
+* Clarify how `odp_packet_concat()` function operates with packets that
+participate in packet data sharing.
+* Clarify that `odp_packet_copy()` function returns always a normal packet, even
+if the source packet is a referencing packet.
+* Change `odp_packet_free_ctrl_set()` function specification to allow usage with
+packets that participate in packet data sharing.
+* Clarify that even though `odp_packet_l2/l3/l4_offset_set()` functions require
+the offset not exceed packet data length, later packet truncation and pulling
+may result in the offset exceeding packet data length.
+* Clarify that `odp_packet_l2/l3/l4_ptr()` pointer value is undefined when
+l2/l3/l4 offset metadata of a packet is larger than the length of the packet
+e.g. after truncating the packet.
+* Fix `odp_packet_head()` specification to not refer to `odp_packet_l2_ptr()`
+as data start.
+
+#### Queue
+* Add `odp_queue_len()` function for reading the number of events currently in
+the queue, or in case of an aggregator queue handle, the number of events
+currently waiting for vector generation.
+* Add supported statistics counters (`odp_queue_stats_opt_t`) for each plain
+queue type into `odp_queue_capability_t`.
+
+#### Scheduler
+* Add `odp_schedule_capability_t.queue_stats` capability for supported
+statistics counters.
+
+#### Timer
+* Add `odp_timeout_is_periodic()` function for checking if a timeout is from a
+periodic timer.
+* Add new `odp_timer_pool_param_t.periodic.max_pending_tmo` parameter and
+matching capabilities `odp_timer_capability_t.periodic.min/max_pending_tmo`
+for configuring flow control for periodic timer pools.
+
+### Implementation
+* Optimize implementations for single event alloc functions (other event types
+than packet).
+* Optimize implementations for single event free functions (other event types
+than packet).
+
+### Performance Tests
+
+#### bench_queue
+* Match test results output to API function names also in throughput mode.
+* Add missing test for `odp_schedule_multi_wait()` function.
+
+#### bench_packet
+* Add test for `odp_packet_is_referencing()` function.
+
+#### bench_timer
+* Add tests for `odp_timeout_is_periodic()` function.
+
+#### pool_perf
+* Add support for testing single event `odp_buffer_alloc/free()` and
+`odp_packet_alloc/free()` functions by setting burst size to zero
+(`--burst 0`).
+* Change default burst size to 32.
+
+#### sched_pktio
+* Decrease packet pool size to 16k to reduce memory usage.
+
 ## OpenDataPlane (1.49.0.0)
 
 ### Backward compatible API changes

config/odp-linux-dpdk.conf

@@ -97,6 +97,10 @@ pktio: {
 }
 
 # DPDK pktio options
+#
+# All options can also be driver specific. If a driver specific option is
+# defined, it overrides the matching generic option. Driver specific options are
+# defined in sections named according to DPDK PMD names (e.g. net_ixgbe).
 pktio_dpdk: {
 	# Default options
 
@@ -118,7 +122,7 @@ pktio_dpdk: {
 	# Enable multi segment transmit offload (RTE_ETH_TX_OFFLOAD_MULTI_SEGS)
 	tx_offload_multi_segs = 0
 
-	# Driver specific options (use PMD names from DPDK)
+	# ixgbe driver specific options
 	net_ixgbe: {
 		rx_drop_en = 1
 	}

config/odp-linux-generic.conf

@@ -122,6 +122,10 @@ pktio: {
 }
 
 # DPDK pktio options
+#
+# All options can also be driver specific. If a driver specific option is
+# defined, it overrides the matching generic option. Driver specific options are
+# defined in sections named according to DPDK PMD names (e.g. net_ixgbe).
 pktio_dpdk: {
 	# Default options
 	num_rx_desc = 128
@@ -139,7 +143,7 @@ pktio_dpdk: {
 	# values. Use 0 for implementation defaults.
 	min_rx_burst = 0
 
-	# Driver specific options (use PMD names from DPDK)
+	# ixgbe driver specific options
 	net_ixgbe: {
 		rx_drop_en = 1
 	}

configure.ac

@@ -3,7 +3,7 @@ AC_PREREQ([2.5])
 # ODP API version
 ##########################################################################
 m4_define([odp_version_generation], [1])
-m4_define([odp_version_major],     [49])
+m4_define([odp_version_major],     [50])
 m4_define([odp_version_minor],      [0])
 m4_define([odp_version_patch],      [0])
 

doc/users-guide/users-guide-packet.adoc

@@ -322,6 +322,7 @@ function like:
 int xmit_pkt(odp_pktout_queue_t queue, odp_packet_t pkt)
 {
 	odp_packet_t ref = odp_packet_ref_static(pkt);
+	/* send through a pktio that supports static packet references */
 	return ref == ODP_PACKET_INVALID ? -1 : odp_pktout_send(queue, ref, 1);
 }
 -----
@@ -330,7 +331,8 @@ This transmits a reference to `pkt` so that `pkt` is retained by the caller,
 which means that the caller is free to retransmit it if needed at a later
 time. When a higher level protocol (_e.g.,_ receipt of a TCP ACK packet)
 confirms that the transmission was successful, `pkt` can then be discarded via
-an `odp_packet_free()` call.
+an `odp_packet_free()` call. The pktio through which the reference is
+sent must support sending static packet references.
 
 The key characteristic of a static reference is that because there are
 multiple independent handles that refer to the same packet, the caller should
@@ -349,8 +351,9 @@ int odp_packet_has_ref(odp_packet_t pkt);
 -----
 
 that indicates whether other packets exist that share bytes with this
-packet. If this routine returns 0 then the caller can be assured that it is
-safe to modify it as this handle is the only reference to the packet.
+packet. If this routine returns 0 and the packet is not a dynamic reference,
+then the caller can be assured that it is safe to modify the packet as this
+handle is the only reference to the packet.
 
 ==== Dynamic References
 While static references are convenient and efficient, they are limited by the
@@ -380,12 +383,19 @@ image::ref.svg[align="center"]
 Following a successful reference creation, the bytes of `pkt` beginning at
 offset `offset` are shared with the created reference. These bytes should be
 treated as read only since multiple references point to them. Each reference,
-however still retains its own individual headroom and metadata that is not
-shared with any other reference. This allows unique headers to be created by
-calling `odp_packet_push_head()` or `odp_packet_extend_head()` on either
-handle. This allows multiple references to the same packet to prefix unique
-headers onto common shared data it so that they can be properly multicast
-using code such as:
+however, has its own metadata that is not shared with any other packet.
+
+Packet data and layout of the referenced packet `pkt` must not be
+modified as long as references to the packet exist. Packet layout of the
+referencing packet, `ref_pkt`, may be modified. The push and extend
+variants of packet manipulation functions always add non-shared data
+to a dynamic reference and pull and trunc functions remove any kind
+(shared or non-shared) data from the reference. Non-shared packet data
+of the referencing packet may be modified.
+
+Prefixing dynamic references with non-shared packet data makes it possible to
+have multiple packets that prefix the same packet data with unique headers,
+which enables multicast without packet data copying using code such as:
 
 [source,c]
 -----
@@ -401,6 +411,10 @@ int pkt_fanout(odp_packet_t payload, odp_queue_t fanout_queue[], int num_queues)
 Receiver worker threads can then operate on each reference to the packet in
 parallel to prefix a unique transmit header onto it and send it out.
 
+Dynamic references and referencing packets can be sent to a pktio only
+if the pktio advertises the capability to send such packets or if the
+packets are sent with the `ODP_PACKET_FREE_CTRL_DONT_FREE` option.
+
 ==== Dynamic References with Headers
 The dynamic references discussed so far have one drawback in that the headers
 needed to make each reference unique must be constructed individually after
@@ -457,7 +471,7 @@ Here two separate header packets are prefixed onto the same shared packet, each
 at their own specified offset, which may or may not be the same. The result is
 three packets visible to the application:
 
-* The original `pkt`, which can still be accessed and manipulated directly.
+* The original `pkt`, which can still be accessed as read only.
 * The first reference, which consists of `hdr_pkt1` followed by bytes
 contained in `pkt` starting at `offset1`.
 * The second reference, which consists of `hdr_pkt2` followed by bytes
@@ -468,10 +482,9 @@ references exist.
 
 ===== Data Sharing with References
 Because a `pkt` is a shared object when referenced, applications must observe
-certain disciplines when working with them. For best portability and
-reliability, the shared data contained in any packet referred to by references
-should be treated as read only once it has been successfully referenced until
-it is known that all references to it have been freed.
+certain disciplines when working with them. Shared data contained in any
+packet should be treated as read only and all packet data of packets referenced
+by other packets should be treaded as read only.
 
 To assist applications in working with references, ODP provides the additional
 API:
@@ -481,24 +494,23 @@ API:
 int odp_packet_has_ref(odp_packet_t pkt);
 -----
 The `odp_packet_has_ref()` API says whether any other packets
-exist that share any bytes with this packet.
+exist that reference packet data of this packet.
 
-===== Compound References
-Note that architecturally ODP does not limit referencing and so it is possible
-that a reference may be used as a basis for creating another reference. The
-result is a _compound reference_ that should still behave as any other
+[source,c]
+-----
+int odp_packet_is_referencing(odp_packet_t pkt);
+-----
+The `odp_packet_is_referencing()` API says whether this packet is a dynamic
 reference.
 
-As noted earlier, the intent behind references is that they are lightweight
-objects that can be implemented without requiring data copies. The existence
-of compound references may complicate this goal for some implementations. As a
-result, implementations are always free to perform partial or full copies of
-packets as part of any reference creation call.
+===== Compound References
 
-Note also that a packet may not reference itself, nor may circular reference
-relationships be formed, _e.g.,_ packet A is used as a header for a reference
-to packet B and B is used as a header for a reference to packet A.  Results
-are undefined if such circular references are attempted.
+Static packet reference may be created to a packet it is already a static
+packet reference. Static packet reference may not be created to a packet
+that is a dynamic reference or a packet that is referenced by a dynamic
+reference. Dynamic references may not be created to packets that are
+static or dynamic references, but may be created to packets that are
+already referenced by another dynamic reference.
 
 === Packet Parsing, Checksum Processing, and Overrides
 Packet parsing is normally triggered automatically as part of packet RX

example/ipsec_crypto/odp_ipsec_cache.c

@@ -72,7 +72,6 @@ int create_ipsec_cache_entry(sa_db_entry_t *cipher_sa,
 	params.op = (in) ? ODP_CRYPTO_OP_DECODE : ODP_CRYPTO_OP_ENCODE;
 	params.op_type = ODP_CRYPTO_OP_TYPE_BASIC;
 	params.auth_cipher_text = TRUE;
-	params.output_pool = ODP_POOL_INVALID;
 	if (CRYPTO_API_SYNC == api_mode) {
 		params.op_mode = ODP_CRYPTO_SYNC;
 		params.compl_queue = ODP_QUEUE_INVALID;