# find_matching_provider: Unbounded Iteration and Weight Underpricing ## Context `find_matching_provider` (`pallet/src/lib.rs:3441-3512`) selects the lowest-priced provider that satisfies a set of storage requirements. It is invoked from the `create_bucket_with_storage` extrinsic (`pallet/src/lib.rs:1271`), which auto-matches a caller to a provider and opens an agreement in a single call. ```rust #[pallet::call_index(16)] #[pallet::weight(T::WeightInfo::create_bucket_with_storage())] pub fn create_bucket_with_storage( origin: OriginFor<T>, max_bytes: u64, duration: BlockNumberFor<T>, max_price_per_byte: BalanceOf<T>, ) -> DispatchResult { // ... Self::find_matching_provider(max_bytes, duration, max_price_per_byte)?; // ... } fn find_matching_provider( bytes_needed: u64, duration: BlockNumberFor<T>, max_price_per_byte: BalanceOf<T>, ) -> Result<(T::AccountId, ProviderInfo<T>), DispatchError> { // ... for (account, info) in Providers::<T>::iter() { // The function iterates the entire `Providers` storage map, // filtering each entry by deregistration state, acceptance flag, // duration,price, capacity, and stake, and tracks the best match // by lowest price. } // ... } ``` ## Problem 1. Unbounded iteration in an extrinsic. `Providers::<T>::iter()` performs a full scan of the provider map, with one storage trie read per provider. 2. Weight does not account for the iteration correctly. The extrinsic is annotated with `T::WeightInfo::create_bucket_with_storage()`, and the corresponding benchmark (`pallet/src/benchmarking.rs:281-298`) registers exactly one provider. The generated weight therefore prices the call as a single-provider scan. As the provider set grows, every caller pays the same fixed fee while the runtime performs O(N) storage reads. 3. No short-circuit. Because selection is "best by lowest price," the function must examine every provider and cannot terminate early if and only if a viable match is found. ## Solution 1. Bound the provider set. Convert `Providers` to a `CountedStorageMap`, add a `MaxProviders` cap, and parameterize the benchmark/weight by the provider count. **=> Not an ideal solution, because we may want to allow more and more providers as possible for further system scaling.** 2. Move matching off-chain; pass the chosen provider in as an argument. The runtime API `find_matching_providers` already exists for off-chain discovery, the caller can selects a provider off-chain by reading state, and `create_bucket_with_storage` takes an explicit `provider` account. The pallet then performs an O(1) lookup of that single provider and re-validates all constraints before opening the agreement. Weight becomes constant and accurate, and the provider set can grow without bound. **=> However, this is more complex client flow and the need for careful on-chain re-validation, since the caller's choice is now untrusted input. Overal, this can be a solid choice** 3. Index sorted set of avaialable providers so matching reads only candidates, not all providers. Replace the full scan with secondary indexes keyed on the matchable dimensions — for example a `StorageMap`/`StorageDoubleMap` bucketing providers by price tier and/or accepting status, or a sorted structure keyed by `price_per_byte`. The extrinsic then iterates only the relevant slice. Weight is parameterized by the size of that slice rather than the whole set. **=> When we need to rebalance the storage due to imbalance of secondary index, or on every provider mutation, the write cost and complexity is still problematic.** 4. Two-phase request/accept instead of atomic auto-match. **=> Like the `create_bucket` extrinsic, so we are no longer need this extrinsic???.**