Apply clipping and snapping to forecast inputs (regressors and target), not just outputs

[BelhsanHmida]

What

lower, upper and snap only shape the forecast output, right before it is stored.
The same operation is wanted on the way in, as a data-cleaning step: a sensor with
implausible spikes or error sentinels should be cleaned up before the model trains on
it, without having to fix the data upstream or copy it to a second sensor.

This applies to regressors and to the target.

Current behaviour

• lower, upper and snap are flat top-level fields on
  TrainPredictPipelineConfigSchema (flexmeasures/data/schemas/forecasting/pipeline.py:240-264),
  validated at load time in validate_post_processing (same file, line 370).
• They are applied in one place only: apply_forecast_post_processing
  (flexmeasures/data/models/forecasting/utils.py:92), called from predict.py:282
  after the model has run.
• Inputs are loaded per entry in load_data_all_beliefs
  (flexmeasures/data/models/forecasting/pipelines/base.py:404) with no value-level
  cleanup. Gap filling follows in detect_and_fill_missing_values (base.py:1113).

Proposed shape

Bounds attach per entry, alongside the existing source filters, with bare sensor IDs
still accepted as they are now:

{
  "regressors": [
    2094,
    {"sensor": 2095, "lower": "0 kW", "snap": {"0 kW": ["0 kW", "0.5 kW"]}}
  ]
}

Regressor entries already accept either an ID or a dict, via SensorIdOrReferenceField
(flexmeasures/data/schemas/sensors.py:1172), so this is additive and backwards
compatible.

One thing to settle: that field deserialises through SensorReferenceSchema
(sensors.py:1079), which is shared with flex-model and flex-context, where these keys
would mean nothing — its default field already documents that kind of wart (line 1120).
Suggest a forecaster-specific subclass carrying the bounds, rather than adding them to
the shared schema.

The target needs a prior change

The target cannot take this shape today. It is sensor = SensorIdField(...) in
ForecasterParametersSchema (pipeline.py:428) — a bare ID, in the parameters schema,
while regressors are references in the config schema. So:

• This depends on converting the target to a reference field, which
  issue-forecast-target-source-filter.md already proposes as its first step. Worth
  doing that one first, or folding the two together.
• Target bounds would then sit in parameters while regressor bounds sit in config. That
  split is inherited, but it is worth confirming we are happy with it.
• When the target is also listed as a past regressor (the autoregressive column), the
  same cleanup must reach both, or the model sees two versions of the same reading.

Order: clip after filling

Bound the values after detect_and_fill_missing_values (base.py:1113).

Caveat to decide on: an out-of-bounds value is not neutralised until after it has been
used to interpolate its neighbours. Given 10, -9999, [missing], 14 and lower: 0, the
gap interpolates between -9999 and 14 to roughly -4993, and then both clip to 0 — so a
gap that should have filled to about 12 becomes 0.

If the purpose is cleaning genuinely bad readings, the more honest operation may be to
treat out-of-bounds values as missing and let interpolation fill across them, rather
than clipping them to the bound and injecting a fake 0 into the training labels. That
interacts with the missing_threshold accounting (base.py:1154), so it is a separate
decision, but it is the one that matches the stated intent most closely.

Work

1. Add the bounds to a forecaster-specific reference schema, and extend
   validate_post_processing (pipeline.py:370) so unparseable quantities still fail at
   load time.
2. Add a helper alongside apply_forecast_post_processing that bounds a single named
   column against a given sensor unit, reusing _parse_snap_intervals (utils.py:57) so
   the [first, second) and reversed-bounds rules stay identical to the output side.
3. Apply it after detect_and_fill_missing_values, per entry.
4. Tests beside the existing ones (flexmeasures/data/tests/test_forecasting_pipeline.py:675-880):
   regressor units differing from the output sensor unit, target and autoregressive
   column agreeing, interaction with gap filling.
5. Docs (documentation/features/forecasting.rst:66-92) and a changelog entry. The docs
   should be explicit that input and output bounds are configured separately and may
   differ — input bounds clean the training data, output bounds shape what is stored.

Note

Like the existing bounds, these would be config-only (no CLI flags), passed via
--config or the API config payload.

ensure_positive (pipeline.py:228) already clips negatives to zero inside the model
(custom_models/base_model.py:151). Before adding a third bounding mechanism, worth
deciding whether it should be deprecated in favour of the explicit bounds.