# Perception Pipeline & Subsystem Contracts

> The 5-stage target identification pipeline covering raycast FOV occlusion, Levenshtein fuzzy license plate OCR, appearance verification, temporal tracking, and the abstract ISLAMProvider interface contract decoupling Swarm from SLAM.

- Repository: ivaziz/swarm_car_search
- GitHub: https://github.com/ivaziz/swarm_car_search
- Human wiki: https://grok-wiki.com/public/wiki/ivaziz-swarm-car-search-66a3d00cb601
- Complete Markdown: https://grok-wiki.com/public/wiki/ivaziz-swarm-car-search-66a3d00cb601/llms-full.txt

## Source Files

- `interfaces/slam_interface.py`
- `interfaces/perception_interface.py`
- `perception/matching_engine.py`
- `perception/vehicle_detector.py`
- `perception/plate_recognizer.py`
- `swarm/communication/local_comm.py`

---

<details>
<summary>Relevant source files</summary>
The following files were used as context for generating this wiki page:
- [interfaces/slam_interface.py](interfaces/slam_interface.py)
- [interfaces/perception_interface.py](interfaces/perception_interface.py)
- [perception/matching_engine.py](perception/matching_engine.py)
- [perception/vehicle_detector.py](perception/vehicle_detector.py)
- [perception/plate_recognizer.py](perception/plate_recognizer.py)
- [perception/appearance_verifier.py](perception/appearance_verifier.py)
- [perception/temporal_tracker.py](perception/temporal_tracker.py)
- [slam_providers/base_slam_provider.py](slam_providers/base_slam_provider.py)
- [swarm/communication/local_comm.py](swarm/communication/local_comm.py)
- [swarm/communication/swarm_messages.py](swarm/communication/swarm_messages.py)
- [swarm/core/robot_agent.py](swarm/core/robot_agent.py)
- [ros2_ws/src/swarm_gazebo/swarm_gazebo/slam_bridge.py](ros2_ws/src/swarm_gazebo/swarm_gazebo/slam_bridge.py)
- [ros2_ws/src/swarm_gazebo/swarm_gazebo/perception_bridge.py](ros2_ws/src/swarm_gazebo/swarm_gazebo/perception_bridge.py)
</details>

# Perception Pipeline & Subsystem Contracts

The target search and identification subsystem in **SwarmX** provides an end-to-end vision, verification, and spatial abstraction architecture designed for multi-robot urban search operations. Autonomous search teams face real-world sensing degradations, including optical occlusions from buildings, motion blur, character ambiguities under optical character recognition (OCR), and transient false alarms caused by non-target civilian vehicles. To resolve these challenges reliably without saturating the swarm with premature alarms, SwarmX implements a rigorous **5-Stage Target Identification Pipeline** that progressively filters sensory inputs and fuses detection confidence across geometry, optical text, vehicle appearance, and multi-frame temporal voting.

Equally critical is the architectural decoupling between spatial awareness and multi-robot swarm decision-making. The swarm intelligence subsystem (task allocation, Ant Colony Optimization, stigmergy, and path planning) never interfaces directly with raw sensor hardware (LiDAR, wheel encoders, IMUs) or concrete SLAM nodes. Instead, all spatial localization, occupancy grids, and exploration frontiers flow through the formal **`ISLAMProvider`** abstract interface contract. This structural boundary establishes zero compile-time coupling between the SLAM and Swarm development teams, allowing seamless runtime substitution between mock simulators, Gazebo bridge nodes, and production ROS 2 navigation stacks.

---

## 1. Subsystem Architecture & Interface Contracts

The SwarmX platform enforces strict interface segregation using Python Abstract Base Classes (`abc.ABC`) and immutable data carriers. The codebase divides engineering ownership cleanly: the SLAM and State Estimation layer is maintained independently from Swarm Coordination and Perception.

```mermaid
classDiagram
    class ISLAMProvider {
        <<Abstract>>
        +get_latest_state(robot_id: str) SLAMState*
        +is_healthy(robot_id: str) bool*
    }

    class BaseSLAMProvider {
        #_latest_states: Dict~str, SLAMState~
        #_health_status: Dict~str, bool~
        +update_cached_state(robot_id: str, state: SLAMState)
        +set_healthy(robot_id: str, healthy: bool)
        +get_latest_state(robot_id: str) SLAMState
        +is_healthy(robot_id: str) bool
    }

    class GazeboSLAMBridge {
        +map_width_m: float
        +map_height_m: float
        +resolution: float
        +_on_scan(robot_id, msg)
        +_on_odom(robot_id, msg)
        +get_latest_state(robot_id: str) SLAMState
        +is_healthy(robot_id: str) bool
    }

    class IPerceptionProvider {
        <<Abstract>>
        +detect_vehicles(robot_id: str, camera_frame_id: str) VehicleDetectionEvent*
        +recognize_license_plate(detection: VehicleDetectionEvent) PlateDetectionEvent*
        +match_watchlist(plate_event: PlateDetectionEvent) MatchEvent*
    }

    class IVehicleDetector {
        <<Abstract>>
        +detect_vehicles(robot_id: str, robot_pose: Pose2D, timestamp: float, vehicles: List~TargetVehicle~, env: UrbanEnvironment) List*
    }

    class SimulatedVehicleDetector {
        +max_detection_range: float
        +fov_radians: float
        +base_confidence: float
        +_has_line_of_sight(rx, ry, tx, ty, env) bool
        +detect_vehicles(...) List
    }

    class SimulatedPlateRecognizer {
        +base_ocr_confidence: float
        +max_ocr_distance: float
        +character_noise_rate: float
        +recognize_license_plate(detection, vehicle) PlateDetectionEvent
    }

    class VehicleAppearanceVerifier {
        +color_weight: float
        +shape_weight: float
        +class_weight: float
        +compute_color_similarity(observed_color, target_color) float
        +compute_shape_similarity(observed_ar, target_ar) float
        +verify_appearance(observed_class, observed_color, observed_ar) Tuple
    }

    class MultiFrameTargetTracker {
        +required_confirmations: int
        +association_radius: float
        +ewma_alpha: float
        +tracks: Dict~str, CandidateTrack~
        +add_observation(...) Tuple
        +prune_stale_tracks(current_time, max_age)
    }

    class MatchingEngine {
        +match_threshold: float
        +candidate_threshold: float
        +w_vehicle: float
        +w_plate: float
        +w_ocr: float
        +w_appearance: float
        +w_temporal: float
        +add_target_vehicle(target: TargetVehicle)
        +evaluate_multi_stage_match(plate_event, detection_event) Tuple
        +classify_plate(plate_event, detection_event) Tuple
    }

    ISLAMProvider <|-- BaseSLAMProvider
    BaseSLAMProvider <|-- GazeboSLAMBridge
    IVehicleDetector <|-- SimulatedVehicleDetector
    MatchingEngine o-- VehicleAppearanceVerifier
    MatchingEngine o-- MultiFrameTargetTracker
```

Sources: [interfaces/slam_interface.py:69-100](), [interfaces/perception_interface.py:86-103](), [perception/vehicle_detector.py:33-47](), [perception/matching_engine.py:51-98](), [slam_providers/base_slam_provider.py:10-40]()

---

### The `ISLAMProvider` Contract

The `ISLAMProvider` interface in `interfaces/slam_interface.py` decouples the Swarm Decision Layer from low-level mapping and localization algorithms. A consumer requires only two operations from any SLAM engine:

```python
# interfaces/slam_interface.py
class ISLAMProvider(ABC):
    """
    Abstract Base Class representing the contract that any SLAM provider
    (MockSLAMProvider or real ROS2-backed SLAM) must satisfy.
    """

    @abstractmethod
    def get_latest_state(self, robot_id: str) -> SLAMState:
        """Retrieve the most recent SLAM estimation for the specified robot."""
        pass

    @abstractmethod
    def is_healthy(self, robot_id: str) -> bool:
        """Check if the SLAM tracking and mapping pipeline is healthy."""
        pass
```

Sources: [interfaces/slam_interface.py:69-100]()

#### Core Data Structures

The interface defines four strongly-typed, serialization-safe data structures:

| Data Structure | Immutability | Core Fields | Description |
| :--- | :--- | :--- | :--- |
| **`Pose2D`** | `@dataclass(frozen=True)` | `x: float`, `y: float`, `theta: float` | 2D position and heading angle in radians $[-\pi, \pi]$ within the global reference frame. |
| **`OccupancyGrid2D`** | `@dataclass` | `width: int`, `height: int`, `resolution: float`, `origin: Pose2D`, `data: List[int]` | 1D row-major array representing urban occupancy: `-1` (unexplored/unknown), `0` (free/navigable), and `100` (occupied/obstacle). |
| **`FrontierCluster`** | `@dataclass(frozen=True)` | `cluster_id: str`, `centroid_x: float`, `centroid_y: float`, `size: int`, `bounding_box: Tuple[float, float, float, float]` | Clustered boundary cells separating explored free space from unknown territory, representing candidate exploration targets for swarm dispatch. |
| **`SLAMState`** | `@dataclass` | `robot_id: str`, `timestamp: float`, `pose: Pose2D`, `linear_velocity: float`, `angular_velocity: float`, `occupancy_grid: OccupancyGrid2D`, `frontiers: List[FrontierCluster]`, `exploration_ratio: float`, `navigation_cost_map: Optional[List[float]]` | The complete instantaneous perception and spatial snapshot passed to the robot agent and coordinator per control cycle. |

Sources: [interfaces/slam_interface.py:12-68]()

#### Runtime Provider Polymorphism

SwarmX supports dynamic dependency injection across environments without altering swarm logic:
1. **`MockSLAMProvider`**: Generates synthetic expanding circular occupancy grids, deterministic frontier clusters, and kinematic poses for headless unit testing and high-speed Monte Carlo simulations.
2. **`BaseSLAMProvider`**: Implements thread-safe caching via an internal `_latest_states: Dict[str, SLAMState]` store and health tracking (`is_healthy`).
3. **`GazeboSLAMBridge`**: Extends `ISLAMProvider` and ROS 2 `rclpy.node.Node`, subscribing directly to `/{robot_id}/odom` and LiDAR laser scans, populating real-time occupancy grids and detecting frontier clusters via raycasting.

Sources: [slam_providers/base_slam_provider.py:10-40](), [ros2_ws/src/swarm_gazebo/swarm_gazebo/slam_bridge.py:39-75](), [ros2_ws/src/swarm_gazebo/swarm_gazebo/slam_bridge.py:321-351]()

---

## 2. The 5-Stage Target Identification Pipeline

The visual detection pipeline isolates target vehicles from civilian distractors through five sequential, interdependent processing stages. Rather than relying on a monolithic classification probability, each stage produces a dedicated confidence coefficient ($C_i \in [0.0, 1.0]$) that is preserved in the emitted event for explainability.

```mermaid
flowchart TD
    subgraph S1["Stage 1: Spatial & Geometric Gating"]
        A["Sensor Frame & Robot Pose2D"] --> B{"Distance <= 14.0m?"}
        B -- No --> R1["Reject: Out of Range"]
        B -- Yes --> C{"Bearing within FOV 90°?"}
        C -- No --> R2["Reject: Out of View"]
        C -- Yes --> D{"Bresenham Raycast: Grid != 100?"}
        D -- No --> R3["Reject: Wall Occlusion"]
        D -- Yes --> E["Emit VehicleDetectionEvent (C_veh)"]
    end

    subgraph S2["Stage 2: Plate Localization"]
        E --> F["Regress Plate Sub-BoundingBox2D"]
        F --> G["Compute Plate Likelihood (C_plate)"]
    end

    subgraph S3["Stage 3: OCR & String Distance"]
        G --> H["Crop Plate Region & Attenuate with Distance"]
        H --> I["Optical Confusion Substitution (8<->B, 0<->O, etc.)"]
        I --> J["Levenshtein Distance vs Watchlist (S_plate)"]
        J --> K["OCR Confidence (C_ocr x S_plate)"]
    end

    subgraph S4["Stage 4: Appearance Verification"]
        E --> L["Extract Class, Color RGB, & Aspect Ratio"]
        L --> M["Class Concordance (1.0 vs 0.2)"]
        L --> N["Euclidean RGB Distance Normalization"]
        L --> O["Aspect Ratio Geometry Concordance"]
        M & N & O --> P["Composite Appearance Score (C_app)"]
    end

    subgraph S5["Stage 5: Multi-Frame Temporal Tracking"]
        K & P --> Q["Associate Spatial Track (< 3.0m Radius)"]
        Q --> R["EWMA Filter on Instantaneous Score (alpha=0.40)"]
        R --> S["Modal Plate Majority Voting (S_modal)"]
        S --> T["Temporal Confidence Calculation (C_temp)"]
    end

    subgraph S6["Fusion & Decision Gate"]
        E & G & K & P & T --> U["Weighted Fusion: C = Sum(w_i * C_i)"]
        U --> V{"Classification Check"}
        V -- "C >= 0.75 & S_plate >= 0.45 & k >= k_req" --> CONF["CONFIRMED MATCH\nTrigger Alarm, Stop, Broadcast"]
        V -- "C >= 0.40 & S_plate >= 0.45" --> CAND["CANDIDATE SIGHTING\nNavigate Closer for High-Res OCR"]
        V -- "Otherwise" --> NON["NON-TARGET\nDiscard False Alarm & Resume Search"]
    end
```

Sources: [perception/vehicle_detector.py:49-182](), [perception/plate_recognizer.py:24-87](), [perception/appearance_verifier.py:34-136](), [perception/temporal_tracker.py:84-222](), [perception/matching_engine.py:51-208]()

---

### Stage 1: Raycast FOV & Occlusion-Aware Vehicle Detection

Visual candidate detection is handled by `SimulatedVehicleDetector` in `perception/vehicle_detector.py`. To eliminate unphysical detections through walls, candidates undergo three sequential spatial tests:

1. **Range Test**: Target Euclidean distance $d = \sqrt{(x_v - x_r)^2 + (y_v - y_r)^2}$ must not exceed `max_detection_range` (default $14.0\text{ m}$):
   $$d \le \min(R_{\max}, R_{\text{detection\_radius}})$$
2. **Angular Field-of-View (FoV) Test**: Heading error $\Delta \theta$ relative to the robot's heading $\theta$ must fall within the camera aperture (default $90^\circ$, or $\pm 45^\circ$ from camera center line):
   $$\theta_{\text{bearing}} = \text{atan2}(y_v - y_r, x_v - x_r)$$
   $$\Delta \theta = (\theta_{\text{bearing}} - \theta + \pi) \pmod{2\pi} - \pi$$
   $$|\Delta \theta| \le \frac{\text{FOV}_{\text{radians}}}{2.0}$$
3. **Obstacle Line-of-Sight (LOS) Raycasting**: An occlusion ray is stepped between $(x_r, y_r)$ and $(x_v, y_v)$ through the `UrbanEnvironment` 2D grid at half-cell resolution intervals ($\Delta s = 0.5 \times \text{resolution}$). If any intermediate grid cell contains value `100` (static obstacle/wall), the vehicle is occluded and rejected:

```python
# perception/vehicle_detector.py
def _has_line_of_sight(self, rx: float, ry: float, tx: float, ty: float, env: Optional[UrbanEnvironment] = None) -> bool:
    if env is None:
        return True
    dx, dy = tx - rx, ty - ry
    dist = math.hypot(dx, dy)
    if dist < 1e-3:
        return True
    steps = int(math.ceil(dist / (env.resolution * 0.5)))
    step_x, step_y = dx / steps, dy / steps
    for i in range(1, steps):
        check_x = rx + i * step_x
        check_y = ry + i * step_y
        g = env.world_to_grid(check_x, check_y)
        if g is not None and env.get_cell(g[0], g[1]) == 100:
            return False  # Obstacle blocks line of sight
    return True
```

If the candidate passes all checks, its distance-attenuated detection confidence $C_{\text{veh}}$ is calculated:
$$f_{\text{dist}} = \max\left(0.0, 1.0 - \frac{d}{1.5 \times R_{\max}}\right)$$
$$C_{\text{veh}} = \min\left(0.99, \max(0.50, C_{\text{base}} \times f_{\text{dist}})\right)$$

The detector projects an image bounding box `BoundingBox2D` whose scale shrinks inversely with distance: $\text{scale} = \max(0.15, \min(0.60, 3.0 / \max(1.0, d)))$.

Sources: [perception/vehicle_detector.py:70-162]()

---

### Stage 2: License Plate Localization

Once a vehicle bounding box is confirmed, Stage 2 localizes the rectangular license plate crop within the lower-central segment of the vehicle silhouette (`BoundingBox2D(ymin=0.60, xmin=0.35, ymax=0.80, xmax=0.65)`). The localization probability $C_{\text{plate}}$ is conditioned on distance-dependent optical resolution:

$$C_{\text{plate}} = \begin{cases} 
\min\left(0.98, \max\left(0.85, 0.95 - \frac{d}{20.0}\right)\right) & \text{if } d \le 6.0\text{ m} \\ 
\min\left(0.85, \max\left(0.60, 0.90 - \frac{d}{15.0}\right)\right) & \text{if } 6.0\text{ m} < d \le 10.0\text{ m} \\ 
\max\left(0.35, 0.70 - \frac{d}{12.0}\right) & \text{if } d > 10.0\text{ m} 
\end{cases}$$

Candidate string visibility also exhibits discrete degradation: vehicles beyond $5.0\text{ m}$ only reveal a partial prefix (`plate[:3] + "-???"`), while vehicles beyond $9.0\text{ m}$ yield no legible plate preview, signaling the swarm agent to close range.

Sources: [perception/vehicle_detector.py:146-171](), [perception/plate_recognizer.py:85-87]()

---

### Stage 3: OCR Engine & Levenshtein Fuzzy Alignment

Stage 3 executes character extraction on the cropped license plate image via `SimulatedPlateRecognizer` (`perception/plate_recognizer.py`) and aligns the result against the target watchlist via `MatchingEngine` (`perception/matching_engine.py`).

#### Optical Confusion Simulation

At long distances ($d > 7.0\text{ m}$) or when environmental noise is injected (`character_noise_rate > 0.0`), optical character recognition engines confuse topologically similar glyphs. The recognizer simulates realistic neural network OCR misclassifications using `OCR_CONFUSION_MAP`:

$$\text{Confusion Mappings:}\quad '8' \longleftrightarrow 'B',\quad '0' \longleftrightarrow 'O',\quad '2' \longleftrightarrow 'Z',\quad '5' \longleftrightarrow 'S',\quad '1' \longleftrightarrow 'I'$$

When corruption occurs, the character confidence is discounted: $\text{conf}_{\text{char}} \leftarrow \max(0.35, \text{conf}_{\text{char}} \times 0.70)$.

Sources: [perception/plate_recognizer.py:15-22](), [perception/plate_recognizer.py:67-81]()

#### Levenshtein Fuzzy String Alignment

To prevent typo-induced misidentifications, `MatchingEngine` strips non-alphanumeric punctuation and computes normalized string similarity:

```python
# perception/matching_engine.py
def normalize_plate(plate: str) -> str:
    """Strips hyphens, spaces, and standardizes to uppercase."""
    return "".join(ch for ch in plate.upper() if ch.isalnum())

def levenshtein_similarity(s1: str, s2: str) -> float:
    """Computes normalized string similarity in range [0.0, 1.0] using Levenshtein distance."""
    if s1 == s2:
        return 1.0
    if not s1 or not s2:
        return 0.0
    len1, len2 = len(s1), len(s2)
    dp = [[0] * (len2 + 1) for _ in range(len1 + 1)]
    for i in range(len1 + 1): dp[i][0] = i
    for j in range(len2 + 1): dp[0][j] = j
    for i in range(1, len1 + 1):
        for j in range(1, len2 + 1):
            cost = 0 if s1[i - 1] == s2[j - 1] else 1
            dp[i][j] = min(dp[i - 1][j] + 1, dp[i][j - 1] + 1, dp[i - 1][j - 1] + cost)
    dist = dp[len1][len2]
    return max(0.0, 1.0 - (dist / float(max(len1, len2))))
```

The composite optical score for Stage 3 combines the OCR model confidence and string alignment score:
$$S_3 = C_{\text{ocr}} \times S_{\text{plate}}$$

Sources: [perception/matching_engine.py:17-50](), [perception/matching_engine.py:128-131]()

---

### Stage 4: Vehicle Appearance Verification

The `VehicleAppearanceVerifier` (`perception/appearance_verifier.py`) prevents plate spoofing or OCR false positives by validating the visual signature of the vehicle against the ground-truth `TargetAppearanceProfile`:

$$C_{\text{app}} = w_{\text{color}} S_{\text{color}} + w_{\text{shape}} S_{\text{shape}} + w_{\text{class}} S_{\text{class}}$$

Default feature weights are: $w_{\text{color}} = 0.55$, $w_{\text{shape}} = 0.25$, and $w_{\text{class}} = 0.20$.

```
+---------------------------------------------------------------------------------+
| VehicleAppearanceVerifier Component Breakdown                                  |
+---------------------------------------------------------------------------------+
| 1. Class Match:                                                                 |
|    S_class = 1.0 if observed_class == target_class else 0.20                     |
|                                                                                 |
| 2. Calibrated Color Similarity:                                                 |
|    RGB Euclidean Distance: D_rgb = sqrt(sum((RGB_obs - RGB_tgt)^2))            |
|    Normalized Distance:    d_norm = D_rgb / 441.673   (where 441.673 = sqrt(3*255^2))|
|    Sharpened Similarity:   S_color = max(0.0, 1.0 - d_norm)^1.5                 |
|                                                                                 |
| 3. Shape & Geometry Concordance:                                                |
|    Aspect Ratio (length / width): Sedan=2.2, SUV=1.8, Truck=2.8, Van=1.9        |
|    Aspect Ratio Error:     e_ar = abs(AR_obs - AR_tgt) / AR_tgt                 |
|    Shape Similarity:       S_shape = max(0.0, 1.0 - e_ar)                       |
+---------------------------------------------------------------------------------+
```

Sources: [perception/appearance_verifier.py:12-31](), [perception/appearance_verifier.py:53-136]()

---

### Stage 5: Multi-Frame Temporal Tracking & Consistency

A single video frame is vulnerable to illumination flashes, glints, or momentary occlusions. The `MultiFrameTargetTracker` (`perception/temporal_tracker.py`) enforces multi-frame spatial and temporal persistence.

1. **Spatial Track Gating**: Observations are assigned to an active `CandidateTrack` if Euclidean separation is within `association_radius = 3.0\text{ m}`. The track centroid is updated incrementally:
   $$\bar{x}_{n+1} = \frac{n \cdot \bar{x}_n + x_{\text{obs}}}{n + 1},\quad \bar{y}_{n+1} = \frac{n \cdot \bar{y}_n + y_{\text{obs}}}{n + 1}$$
2. **Instantaneous Frame Scoring**:
   $$\text{score}_{\text{inst}} = 0.20 C_{\text{veh}} + 0.35 (C_{\text{ocr}} \times S_{\text{plate}}) + 0.25 C_{\text{app}} + 0.20 S_{\text{plate}}$$
3. **EWMA Confidence Accumulation**: Instantaneous scores are smoothed over time using Exponential Weighted Moving Average with learning rate $\alpha = 0.40$:
   $$C_{\text{accum}}^{(t)} = (1.0 - \alpha) C_{\text{accum}}^{(t-1)} + \alpha \cdot \text{score}_{\text{inst}}$$
4. **Modal Plate Majority Voting**: Detections within the track vote on the plate string. The modal plate frequency $R_{\text{modal}} = \frac{\text{count}_{\text{modal}}}{k}$ is weighted by its Levenshtein similarity to target plate $S_{\text{modal}}$:
   $$R_{\text{match}} = R_{\text{modal}} \times S_{\text{modal}}$$
5. **Frame Count Factor**: Confidence scales with observation count $k$ up to required confirmations $k_{\text{req}}$:
   $$F_{\text{factor}} = \min\left(1.0, \frac{k}{k_{\text{req}}}\right)$$
6. **Composite Temporal Confidence**:
   $$C_{\text{temp}} = 0.50 C_{\text{accum}} + 0.30 R_{\text{match}} + 0.20 F_{\text{factor}}$$

A track is verified only when $k \ge k_{\text{req}}$, $C_{\text{temp}} \ge 0.82$, and $S_{\text{modal}} \ge 0.75$. Stale unconfirmed tracks older than $60.0\text{ s}$ are pruned via `prune_stale_tracks()`.

Sources: [perception/temporal_tracker.py:84-235](), [perception/temporal_tracker.py:239-247]()

---

## 3. Multi-Stage Fusion & Swarm Decision Logic

The overall target verification confidence $C$ combines all five stages into a single metric computed by `MatchingEngine.evaluate_multi_stage_match()`:

$$C = w_{\text{veh}} C_{\text{veh}} + w_{\text{plate}} C_{\text{plate}} + w_{\text{ocr}} (C_{\text{ocr}} \times S_{\text{plate}}) + w_{\text{app}} C_{\text{app}} + w_{\text{temp}} C_{\text{temp}}$$

```python
# perception/matching_engine.py
composite_confidence = (
    self.w_vehicle * c_veh              # w_vehicle = 0.15
    + self.w_plate * c_plate            # w_plate = 0.15
    + self.w_ocr * ocr_composite        # w_ocr = 0.30, ocr_composite = c_ocr * best_similarity
    + self.w_appearance * c_app         # w_appearance = 0.20
    + self.w_temporal * c_temp          # w_temporal = 0.20
)
composite_confidence = max(0.0, min(0.99, round(composite_confidence, 3)))
```

Sources: [perception/matching_engine.py:71-85](), [perception/matching_engine.py:156-164]()

### Fusion Weights & Stage Diagnostics

| Stage | Metric Symbol | Default Weight | Key Diagnostic Output | Failure Consequence |
| :--- | :---: | :---: | :--- | :--- |
| **1. Vehicle Detector** | $C_{\text{veh}}$ | `0.15` | `stage_breakdown["vehicle_confidence"]` | Wall occlusion or out-of-range leads to immediate frame drop. |
| **2. Plate Localizer** | $C_{\text{plate}}$ | `0.15` | `stage_breakdown["plate_confidence"]` | Low resolution attenuates overall score, forcing closer navigation. |
| **3. Plate OCR & Similarity** | $C_{\text{ocr}} \times S_{\text{plate}}$ | `0.30` | `stage_breakdown["plate_similarity"]` | Character typos mitigated by Levenshtein matching; decoy plates rejected. |
| **4. Appearance Verifier** | $C_{\text{app}}$ | `0.20` | `stage_breakdown["appearance_match"]` | Wrong vehicle color or class penalizes score, stopping spoofed plates. |
| **5. Temporal Tracker** | $C_{\text{temp}}$ | `0.20` | `stage_breakdown["temporal_consistency"]` | Transient single-frame spikes cannot cross confirmation threshold. |

Sources: [perception/matching_engine.py:71-77](), [perception/matching_engine.py:165-175]()

---

### Classification Tiers & FSM State Transitions

The robot agent's Finite State Machine (`swarm/states/state_machine.py`) responds dynamically to the classification returned by `classify_plate()`:

```
               +---------------------------------------+
               |        Perception Scan Result         |
               +---------------------------------------+
                                  |
         +------------------------+------------------------+
         |                                                 |
[C >= 0.75 & S_plate >= 0.45 & k >= k_req]     [C >= 0.40 & S_plate >= 0.45]
         |                                                 |
         v                                                 v
  +--------------+                                  +--------------+
  |  CONFIRMED   |                                  |  CANDIDATE   |
  +--------------+                                  +--------------+
  | 1. Stop Robot Kinematics                        | 1. Lock Waypoint to (cx, cy)
  | 2. FSM -> REPORTING                             | 2. FSM -> NAVIGATING
  | 3. Broadcast Alert to Swarm Bus                 | 3. Close range (< 5.0m)
  | 4. Escalate Mission Completion                  | 4. Re-evaluate with high-res OCR
  +--------------+                                  +--------------+
         |
         | (All other outcomes: C < 0.40 or S_plate < 0.45)
         v
  +--------------+
  |  NON_TARGET  |
  +--------------+
  | 1. Discard Observation / Increment False Positives
  | 2. FSM -> SEARCHING (continue frontier patrol)
  +--------------+
```

```python
# swarm/core/robot_agent.py
if classification == "CONFIRMED" and match_event is not None:
    self.sim_robot.stop()
    self._reporting_elapsed = 0.0
    self.fsm.transition_to(RobotState.REPORTING, timestamp, reason="CONFIRMED_TARGET_MATCH")
    self.last_confirmed_match = match_event
    self.total_target_confirmations += 1
    if self.comm_channel is not None:
        alert_msg = create_vehicle_alert_message(self.robot_id, timestamp, match_event)
        self.comm_channel.broadcast(alert_msg)
    return match_event
elif classification == "CANDIDATE":
    cx, cy = det_event.estimated_global_position
    self.assigned_waypoint = (cx, cy)
    if self.current_state != RobotState.NAVIGATING:
        self.fsm.transition_to(RobotState.NAVIGATING, timestamp, reason="APPROACHING_CANDIDATE_FOR_OCR")
    return None
else:
    self.total_false_positives += 1
    if self.current_state != RobotState.SEARCHING:
        self.fsm.transition_to(RobotState.SEARCHING, timestamp, reason="NON_TARGET_OR_UNCONFIRMED")
    return None
```

Sources: [swarm/core/robot_agent.py:215-248]()

---

## 4. Swarm Communication & Stigmergic Feedback

When a robot confirms or suspects a target, perception events propagate across the multi-robot collective through direct messaging and stigmergic pheromone fields.

```mermaid
sequenceDiagram
    autonumber
    participant Cam as Camera / Detector
    participant Agent as RobotAgent (robot_1)
    participant FSM as StateMachine
    participant Comm as LocalMessageBus
    participant Peer as RobotAgent (robot_2)
    participant Coord as SwarmCoordinator

    Note over Cam,Agent: Step 1-4 Perception Sweep
    Cam->>Agent: VehicleDetectionEvent + PlateDetectionEvent
    Agent->>Agent: evaluate_multi_stage_match()

    alt Classification == CANDIDATE
        Agent->>FSM: transition_to(NAVIGATING, "APPROACHING_CANDIDATE")
        Agent->>Agent: Move closer to candidate (< 5m) for crisp OCR
    else Classification == CONFIRMED
        Agent->>Agent: sim_robot.stop()
        Agent->>FSM: transition_to(REPORTING, "CONFIRMED_TARGET_MATCH")
        Agent->>Comm: broadcast(create_vehicle_alert_message)
        Comm->>Peer: deliver(SwarmMessage: TARGET_FOUND)
        Comm->>Coord: deliver(SwarmMessage: TARGET_FOUND)
        Coord->>Coord: Escalate mission status & halt exploration
        Peer->>Peer: Abort search, transition to rendezvous / assist
    end
```

Sources: [swarm/core/robot_agent.py:215-231](), [swarm/communication/swarm_messages.py:134-150](), [swarm/communication/local_comm.py:28-33]()

### Inter-Robot Alert Message Schema

Direct communication is implemented via `LocalMessageBus` (`swarm/communication/local_comm.py`), which delivers zero-latency in-memory messages. A confirmed sighting triggers `create_vehicle_alert_message()` in `swarm/communication/swarm_messages.py`:

```python
# swarm/communication/swarm_messages.py
def create_vehicle_alert_message(
    sender_id: str,
    timestamp: float,
    match_event: MatchEvent,
) -> SwarmMessage:
    """Constructs a high-priority vehicle detection broadcast message."""
    payload = {
        "target_id": match_event.target_id,
        "plate_number": match_event.plate_number,
        "match_confidence": match_event.match_confidence,
        "confirmed": match_event.confirmed,
        "location": list(match_event.location),
        "evidence_reference": match_event.evidence_reference,
    }
    return SwarmMessage(
        message_id=str(uuid.uuid4()),
        sender_id=sender_id,
        timestamp=timestamp,
        msg_type=MessageType.TARGET_FOUND,
        payload=payload,
    )
```

Sources: [swarm/communication/swarm_messages.py:134-155]()

### Stigmergic Recruitment

In addition to direct network broadcasts, sightings interact with the swarm's spatial pheromone fields:
* **Recruitment Pheromone ($\tau_s$, rate $\rho = 0.05$)**: When a candidate vehicle is detected ($C \ge 0.40$), recruitment pheromone is deposited at the candidate's coordinates $(x_v, y_v)$. Surrounding robots sensing the local gradient redirect their exploration paths toward the candidate to provide multi-angle camera verification.
* **Avoidance & Reservation Pheromone ($\tau_a$, rate $\rho = 0.30$)**: Once verified or during approach, robots deposit reservation pheromone to prevent multiple agents from colliding or redundantly searching the same vehicle spot.

Sources: [swarm/core/robot_agent.py:215-231](), [ros2_ws/src/swarm_gazebo/swarm_gazebo/perception_bridge.py:180-200]()

---

## 5. Summary

The SwarmX perception architecture integrates a rigorous 5-stage target identification pipeline with an abstracted SLAM boundary. By sequentially cascading **raycast field-of-view occlusion gating**, **license plate sub-region localization**, **Levenshtein fuzzy OCR alignment**, **RGB Euclidean appearance verification**, and **multi-frame temporal tracking**, the platform suppresses optical noise and rejects decoy vehicles before escalating mission-critical alerts. Furthermore, decoupling state estimation behind the abstract **`ISLAMProvider`** contract guarantees that swarm navigation and coordination algorithms remain modular and portable across unit testing mocks, 3D Gazebo physics simulations, and physical ROS 2 hardware deployments.