Slide 1: Welcome to Our Webinar: CAP Area Monitoring System - Lessons from Slovenia
DF icacove 10 ose mm i WELCOME TO OUR WEBINAR \ CAP Area Monitoring System - - Lessons from Slovenia WE WILL BE STARTING SHORTLY!
Slide 2: Planet Fusion for Inconclusive Parcels in Slovenia
Planet Fusion for Inconclusive Parcels in Slovenia
Slide 3: Slovenian National Paying Agency (ARSKTRP)
Slovenian National Paying Agency (ARSKTRP) ARSKTRP are among early implementers of a CAP Area Monitoring System (AMS) Preparation for policy change in 2023 requiring wall-to-wall eligibility checks. Basing AMS Copernicus data primarily Sentinel-2. Face challenges with parcels which cannot be automat
Slide 4: Slovenian National Paying Agency (ARSKTRP)
Slovenian National Paying Agency (ARSKTRP) Slovenian CAP info 2021 ● 56.000 individual farmers ● 825.000 agricultural parcels ● Roughly 500.000 ha ● Receiving €225M in CAP subsidies Overall AMS components 1) Automated Checks using ML algorithms 2) Expert Judgment Checks with Photo interpretation 3)
Slide 5: Challenges with Inconclusive Parcels
Challenges with Inconclusive Parcels Small and Narrow parcels Size and geometry pose a challenge when using Sentinel-2 data. Inconclusive by automated checks. JRC definition of small parcel: ➔ at least 8 full pixels inside a border and less than 60% of pixels lost Definition of small parcels Slovenia:
Slide 6: Challenges with Inconclusive Parcels
Challenges with Inconclusive Parcels Small and Narrow parcels Size and geometry pose a challenge when using Sentinel-2 data. Inconclusive by automated checks. JRC definition of small parcel: ➔ at least 8 full pixels inside a border and less than 60% of pixels lost Definition of small parcels Slovenia:
Slide 7: Challenges with Inconclusive Parcels
Challenges with Inconclusive Parcels Missing Events Limitations due to Cloud Cover and Sentinel-2 Cadence Mowing event not detected results in inconclusive parcel in automated checks
Slide 8: Challenges with Inconclusive Parcels
Challenges with Inconclusive Parcels Missing Events Limitations due to Cloud Cover and Sentinel-2 Cadence Mowing event not detected results in inconclusive parcel in automated checks
Slide 9: Challenges with Inconclusive Parcels
Challenges with Inconclusive Parcels Inconclusive Parcels Overview Inconclusive parcels cannot be processed with ML algorithms and require manual follow up. 25% of agricultural parcels cannot be processed by using Sentinel-2 In total 246.000 parcels requiring a manual follow up In preparation for wa
Slide 10: PlanetScope vs. Planet Fusion over Galicia, Spain
PlanetScope vs. Planet Fusion over Galicia, Spain
Slide 11: Planet Fusion
Planet Fusion PlanetScope Harmonised to Sentinel-2 Key Specifications ➔ Daily 3 meter spatial resolution of all EU Member States ➔ Radiometrically Sentinel-2 comparable ➔ Peer reviewed CESTEM Process ➔ Analysis Ready Data (ARD) ➔ Cloud & Gap Free Sentinel-2 Planet Fusion Spatial 10m 3m Resolution Rev
Slide 12: Improved Detection of Events
Improved Detection of Events Mowing Detected in Fusion
Slide 13: Planet Fusion for Expert Judgement Checks
Planet Fusion for Expert Judgement Checks ARSKTRP has to follow up on remaining parcels which have inappropriate agricultural activity or misclassification of ML models Trained personnel carry out Photo interpretation Sentinel-2 Time Series - Not able to identify mowing Extensive Orchard Example
Slide 14: Planet Fusion for Expert Judgement Checks
Planet Fusion for Expert Judgement Checks ARSKTRP has to follow up on remaining parcels which have inappropriate agricultural activity or misclassification of ML models Trained personnel carry out Photo interpretation Planet Fusion Time Series - Mowing Identified Extensive Orchard Example
Slide 15: Planet Fusion for Inconclusive Parcels
Planet Fusion for Inconclusive Parcels Key Results for Automated Checks ➔ Sentinel-2: Inconclusive parcels 246.000 ➔ Planet Fusion: Inconclusive parcels 66.000 Reduction of inconclusive parcels by 76% Cost savings when compared to manual follow-ups around €1m
Slide 16: Planet Fusion for Inconclusive Parcels
Planet Fusion for Inconclusive Parcels Key Results in Expert Judgment Checks Planet Fusion could help in 20% of cases Reduces amount of personnel hours spent on photo-interpretation resulting in savings near €70.000. Qualitative benefits: ➔ Fewer farmer requests for GeoTagged photos, field visits and
Slide 17: CbM in Slovenia
CbM in Slovenia Deep dive grega.milcinski@sinergise.com Sinergise
Slide 18: The Challenge
The Challenge Objective • CbM of Basic Payment Scheme and Less Favorable Areas in 2021 • AMS in 2023 Starting point • Well established LPIS and GSAA with Farmers Block and annual crop classifications • Operational services for EO data and signal processing, visualization (Sentinel Hub) • Semi-operat
Slide 19: Not a rocket science
Not a rocket science It’s not about making pretty maps, but knowing how our maps will help decision making process. And to make sure that these decisions are best as possible and what client wants.
Slide 20: Signal processing
Signal processing processing of data over FOIs • cloud masks AP at St • NDVI, LAI,... I • rasters, statistics Process API W S M T
Slide 21: The curse of small parcels
The curse of small parcels JRC • Internal buffer of 10 meters (or 5m or 14m?) • At least 5 pixels (or 8 pixels) SI AAMRD • No internal buffer • At least 1 pixel
Slide 22: Input data
Input data Satellite data Training and validation data • Sentinel-2, (Sentinel-1) • LPIS • PlanetScope/Planet Fusion • Geospatial Aid Application (declarations) Auxiliary data • Manually labelled data • DOF, VHR (super-resolution?)
Slide 23: Data Clean-up
Data Clean-up s2cloudless observation outlier detector Machine-learning (ML) approach applicable over all land types ⬩ ML approach (simplistic model) available on Sentinel Hub (and open-source) ⬩ based on a small set of labels (~1k FOIs) ⬩ not meant for all land-types (unless supported by labelled d
Slide 24: Homogeneity marker
Homogeneity marker Detecting FOIs with single-crop claims growing multiple crops Useful also for cleaning up crop-related training data Trained a ML model on a set of fields where multiple crops are expected Training sample with multiple crops • different crops share relatively equally large parts ˌ
Slide 25: Homogeneity marker
Homogeneity marker model evaluated on independent part of training sample False Positive accuracy of 90% Model Confidence Non-homogeneous Homogeneous False Negative
Slide 26: Similarity marker
Similarity marker 2021 - Pregled in urejanje semaforjev po zahtevkih sztuecolr— v/ 8] [BG] vaovive /; / 0) iaroon Mi Bd ee ant oe fps KMRS: 209) op ocak Marka Reziala aso» 2 ag Kem: DDogodek Poaronnest Datum Koen oe ode Modal — 805 0-121 Tost — 505 20-121 Medel — 806 26-121 Woasl $05 20-121 adel — 8
Slide 27: Similarity marker
Similarity marker Similarity-based markers: compare how similar is: Similarity marker • FOI to another FOI - metric is reduced chi2 (χ2) • FOI to average of neighbourhood - compare FOI to average of neighbourhood Distance marker - metric is normalized Euclidean dist. k-th obs. of i-th FOI of veg. in
Slide 28: Similarity marker – built-up area
Similarity marker – built-up area DOF Google
Slide 29: Similarity marker – land-use change
Similarity marker – land-use change
Slide 30: Mean-NDVI marker
Mean-NDVI marker simply the average NDVI value in a window of time indicates low vegetation • expected for arable land or greenhouses Average of meadows in the • not expected for meadows, grassland, ... neighbourhood NDVI profile selected FOI
Slide 31: Mean-NDVI marker
Mean-NDVI marker Example claimed to be permanent meadows Average of meadows in the neighbourhood NDVI profile of this FOI
Slide 32: Mean-NDVI marker - pixel level
Mean-NDVI marker - pixel level enables intra-FOI inspection works the same as the FOI-level algorithm, but on pixels
Slide 33: Mean-NDVI marker - pixel level
Mean-NDVI marker - pixel level enables intra-FOI inspection works the same as the FOI-level algorithm, but on pixels
Slide 34: Mean-NDVI marker - pixel level
Mean-NDVI marker - pixel level calculate coverage at NDVI threshold 0.45
Slide 35: Mean-NDVI marker – pixel level
Mean-NDVI marker – pixel level Example claimed to be permanent meadow Mean-NDVI Map - mean-NDVI value below or above threshold (July - October) Mean-NDVI Map below or above Sentinel-2 VHR threshold (July - October)
Slide 36: Bare soil marker
Bare soil marker 2021 - Pregled in urejanje semaforjev po zahtevkih S2Tive eal EET sii iskonie Nostavive Naloge:0144 _———s roots GEEEY € cv retro cane ‘ Kosnia Dogodk 0 _ nid sO so 18 am 0 20414 20520414 5052010 505205101 $05 20-101 Svan2021' Fed War Apr Maj un Jul). (Ot ‘52 NOVI okelica S2LIC Gola
Slide 37: Bare-soil marker - labels
Bare-soil marker - labels manual labeling by domain knowledge of experts (2000 FOIs) • expected sowing period/expected crop presence period/harvest time • positive labels: FOIs with expected BS in the appropriate period • negative labels: FOIs with expected crops in the appropriate period + meadows
Slide 38: Bare-soil marker - results
Bare-soil marker - results decision tree-based model x: efficiency to correctly predict a FOI with BS Optimal point y: efficiency to correctly predict a FOI without BS (1,1) False positives/negatives were inspected • issue is more on the labeling side • models performs better than metrics suggest
Slide 39: Bare-soil marker - pixel-level
Bare-soil marker - pixel-level Bare-soil model has been converted to an Sentinel Hub Custom Script Bare-soil shown with violet overlay
Slide 40: Mowing marker
Mowing marker 2021 - Pregled in urejanje semaforjev po zahtevkin $2 True coor Zemiovid EEEIB) cote emo Sonar Sant Ge oa recut 2 079 2021-02 U, Te Eee S2L1C Kosi Pada 2004-08-40 gael a eee ? TS C q ovo 0422833 - ¥ 4 SL odin: 0.030118 Jan 2001 Feb Mar Maj Jun Sug 32 NOVI okonca S2L1C Gola zemia ‘S2NDY
Slide 41: Mowing marker
M Owl ng m a r e r Mowing Marker V3 - ID 165282 | Label 204 @ = NDVI data aren) 0.8 | — trend mean 4/- st.dev. abel) time range of interest, mowing event
Slide 42: Mowing marker - false positives
Mowing marker - false positives missed clouds cloud shadow haze
Slide 43: Mowing marker - false negatives
Mowing marker - false negatives mowing not detected if: • meadow has not been mowed (obvious, not a false) • no valid observation around mowing event • tree canopy covers grass beneath (extensive orchard) • NDVI drop not obvious • partial mowing • grazing no algorithm can reduce this number (FN) to
Slide 44: Mowing marker – clouded areas
Mowing marker – clouded areas Planet Fusion Sentinel- 2
Slide 45: Mowing marker – partial mowing
Mowing marker – partial mowing Mowing event No valid observations before mowing event Partial mowing event
Slide 46: Mowing marker - pixel-level
Mowing marker - pixel-level • the same algorithm can be applied on the pixel-level • increase NDVI drop threshold to compensate for fluctuations • enables detection of partial mowing
Slide 47: Crop-related markers
Crop-related markers Claimed corn, similar to winter wheat
Slide 48: Crop-related markers
Crop-related markers Similarity-based markers: ML-based marker: compare how similar is: ⬩ requires grouping and training • FOI to another FOI ⬩ low interpretability • FOI to average of neighbourhood ⬩ high accuracy fast results ⬩ quality of output depends on quality of input no training or grouping
Slide 49: Crop group marker
Crop group marker Similarity-related markers can be used as • early identification of mistakes in the claim • means of reducing label noise in training data Declarations from past years taken into account for crop groups with few samples ML-based crop group classifier works best for checking FOIs co
Slide 50: Crop group classification
Crop group classification Sentinel-2 accuracy: 93.9 Weighted average • Precision: 93.4 • Recall: 93.9 • F1-score: 93.6
Slide 51: Land-use group marker
Land-use group marker very similar to crop-type group classifier same model, same approachParcel boundary delineation Weighted average (precision, recall, F1-score) (89.0, 89.5, 88.9) (92.9, 93.0, 92.8) (89.8, 90.2, 89.8)
Slide 52: Built-up area marker
Built-up area marker ML trained on OSM/national data Pixel based Converted to Sentinel Hub Custom script to allow for Signal processing
Slide 53: Basics
Basics Simple methods are often as good (or better) than fancy ML Try to understand, what is happening within the marker Labels are key • Training and validation • Automatic and manual Understand your signal Credit: David Kidd
Slide 54: Traffic Light System
Traffic Light System Business rules No more EO data, Thresholds vectors, raster Spreadsheet-like
Slide 55: Expert judgement application
Expert judgement application
Slide 56: Communication with farmers
REPUBLIKA oa 8 SLOVENIA Grega Milcinski dnja posodobitey rezultat APLIKA LOG APM Selene MCN SMar Datum zadnjega prevzema podatkov iz geoprostorsk Stanje semaforja po poljinah (63/63) Q_ Iskanje [=] YY Filtriraj po stanju semaforja: 1@ [vee (0) GERK 1070113 - ZA GABROVSKOM 2 6724627 - travnodeteljne
Slide 57: Communication with farmers
Communication with farmers test-expert | Odjava Zadnja posodobitev rezultatov: 3. 11. 2021 Daturn zadnjega prevzema podatkov iz geoprostorskega obrazca: 4. 11. 2021 Stanje semaforja po poljinah (6/6) Q Iskanje [=] Y Filtriraj po stanju semaforja: [0 | v@ || x0 GERK 1719782 - TRAVNIK DOMA .y . 2 2 .
Slide 58: 2021 campaign results
2021 campaign results End of October 7% yellows (3% after financial threshold) In following months down to 4% (1%) 600 red (<0.1%) + 3700 yellow (0.5%)
Slide 59: Experience with Planet Fusion
Experience with Planet Fusion
Slide 60: Planet Fusion performance
Planet Fusion performance Test on 10.000 parcels using S2 and PF • ”Is there mowing event where there should be one” • 96% detected by S2, detected also by PF • 54% of those missed by S2 were detected by PF • 4% of events detected by S2 were missed by PF
Slide 61: Planet Fusion performance
Planet Fusion performance 00 Mowing Marker - ID 11193824 | Label 204 | Source $2 Mowing Marker - ID 11193824 | Label 204 | Source PF @ NDVidata time range of interest Parameter ldrop thr = 0.100 lgrow_r = 1.000 [riter= (CLP <= 0.99 or CLM <= 0.91) and OUT PROBA <= 0.125 02 @ NDVI data time range of
Slide 62: Sentinel-2, Planet Fusion and PlanetScope
Sentinel-2, Planet Fusion and PlanetScope
Slide 63: Planet data experience
Planet data experience Both Planet Fusion and PlanetScope are useful ML models designed on Sentinel-2 perform well on Planet Fusion • Re-training is needed (different bands, atmospheric correction) Added value • Small parcels • Cloudy areas (optical data, Sentinel-1B issue) • Cost of data vs. cost o
Slide 64: Lessons learned
Lessons learned
Slide 65: Lessons learned – mowing through time
Lessons learned – mowing through time By beginning of August, in 80% of the FOIs, where mowing is about to happen, it will happen
Slide 66: Lessons learned – mowing event detection
Lessons learned – mowing event detection To reliably detect a mowing event, an observation a week is sufficient. • Two consecutive cloudy S2 observations might miss it • Planet Fusion to the rescue (Second OpinionTM)
Slide 67: Lessons learned – crop-type grouping
Lessons learned – crop-type grouping More groups is not a recipe for better results • Large mixing between groups • Less confidence in specific prediction (thresholds) • Not necessary for BPS • For future needs, create different models
Slide 68: Lessons learned – validation and labels
Lessons learned – validation and labels Validation of individual markers is difficult, but it is possible • declarations are not (always) ground truth • check for both true positives and true negatives • 3rd party sources are plentiful • buildings, roads, forest registry, OSM • be careful about nois
Slide 69: Summary
Summary Tooling for the system is readily available Iterative process (run, analyse, discuss with PA, understand the impact, revise, run…) Manual analysis and interpretation is key, customized tools are essential Understand your system AMS and CBM are similar process-wise There is still time to set-
Slide 70: More info
More info https://medium.com/sentinel-hub/area-monitoring/home
Slide 71: Thank you for attending our webinar
£ d G/ TF A if re Le 4 Lh Z ea you for attending > our webinar! WANT. TO LEARN MORE? GET IN TOUCH! TOR NIELSEN ») GREGA MILCINSKI Applications in Ag Policy and - : CEO and co-founder, Education & Research, Planet Sh Sinergise tor@planet.com o y, grega.milcinski@sinergise.com