echogram

Echogram Visualisation and Analysis

Easily import multi-frequency acoustic data stored in 'HAC' files (see http://biblio.uqar.ca/archives/30005500.pdf for more information on the format), and produce echogram visualisations with predefined or customized color palettes. It is also possible to merge consecutive echograms; mask or delete unwanted echogram areas; model and subtract background noise; and more important, develop, test and interpret different combinations of frequencies in order to perform acoustic filtering of the echogram's data.

Comprehensive Review

Version: 1.0.1 | Author: Héctor Villalobos | License: GPL-3

Overview

The echogram package provides tools for importing, visualising, and analysing multi-frequency acoustic data from scientific echosounders (Simrad EK60). It supports two input formats:

1. HAC files — the ICES standard data exchange format (HAC specification)
2. RAW files — the native binary format produced by Simrad EK60 echosounders

The package produces echogram visualisations (colour-coded 2D images of acoustic backscatter as a function of depth and ping/time/distance), and enables multi-frequency analysis such as acoustic filtering by combining data from different frequencies.

---

Core Data Structure: "echogram" class

Almost every function in the package operates on or produces an object of class "echogram", which is a list with three components:

Component	Type	Description
depth	numeric vector (length p)	Mean depth (m) of each sample row
Sv	matrix (p × k)	Mean volume backscattering strength (Sv, in dB); has an "frequency" attribute (e.g. "38 kHz")
pings	data.frame (k rows)	Per-ping metadata: pingTime, detBottom, speed, cumdist

---

Architecture & Function Relationships

The package is organised into four functional layers, shown below from bottom (low-level I/O) to top (user-facing analysis).

graph TD
    subgraph "Layer 1: Low-level Binary I/O"
        A["read.EK_raw"]
        B["get_dgIdx"]
        C["dgTime"]
        D["xcvrConf"]
    end

    subgraph "Layer 2: Datagram Parsers"
        E["get_CON0"]
        F["get_NME0"]
        G["get_RAW0"]
        H["parse.nmea"]
    end

    subgraph "Layer 3: File Readers → echogram objects"
        I["read.EK60_raw"]
        J["read.EK_bot"]
        K["read.EK_idx"]
        L["read.echogram"]
        M["ek2echogram"]
        N["sampleRange"]
        O["convertPower"]
        P["convertAngles"]
    end

    subgraph "Layer 4: Visualisation & Analysis"
        Q["echogram"]
        R["palette.echogram"]
        S["join.echogram"]
        T["match.echogram"]
        U["add.echogram"]
        V["mask.echogram"]
        W["trim.echogram"]
        X["sample.echogram"]
        Y["noise.echogram"]
    end

    subgraph "HAC helpers"
        L1["position.hac"]
        L2["navigation.hac"]
        L3["bottom.hac"]
        L4["mergeSvmat"]
    end

    A --> B
    A --> E
    A --> F
    A --> G
    A --> J
    A --> K
    B --> E
    B --> F
    B --> G
    B --> J
    B --> K
    C --> E
    C --> F
    C --> G
    C --> J
    C --> K
    D --> E
    E --> I
    F --> I
    G --> I
    H --> I
    I --> M
    N --> M
    N --> O
    O --> M
    P -.-> M
    L1 --> L
    L2 --> L
    L3 --> L
    L4 --> L
    L4 --> S
    R --> Q
    W --> Q
    Q --> X
    T --> U
    V --> U

Loading

---

Function-by-Function Reference

Layer 1 — Low-level Binary I/O

read.EK_raw

Reads an entire EK60/EK80 file as a raw byte vector. Foundation for all RAW-file processing. Not intended to be called directly.

get_dgIdx

Scans the raw byte vector and builds a datagram index table — a data.frame recording where each datagram (CON0, NME0, RAW0, BOT0, IDX0, etc.) starts, ends, and its length. This index is the key for all subsequent datagram extraction.

dgTime

Extracts the datagram timestamp from 8 raw bytes at a given offset. Converts the Windows FILETIME (100-nanosecond intervals since 1601-01-01) to POSIXct in UTC.

xcvrConf

Parses 320 bytes of transceiver configuration data: channel ID, beam type, frequency, gain, equivalent beam angle (EBA), beamwidths, angle sensitivities/offsets, position, direction, and calibration tables (pulse lengths, gains, Sa corrections). Called by get_CON0 once per transceiver.

---

Layer 2 — Datagram Parsers (EK60 RAW format)

get_CON0

Extracts the CON0 configuration datagram: header info (survey name, transect name, sounder name, software version, transceiver count) and transceiver configuration for each channel.

• Calls: read.EK_raw → get_dgIdx → dgTime, xcvrConf

get_NME0

Extracts all NME0 datagrams (NMEA sentences) — GPS strings with timestamps. Returns a data.frame of dgTime and raw NMEA text strings.

• Calls: read.EK_raw → get_dgIdx → dgTime

get_RAW0

Extracts all RAW0 datagrams containing the actual acoustic data:

• A 3D sampleData array [pings × 21 fields × transceivers] with per-ping metadata (channel, frequency, power, pulse length, sound velocity, absorption, heave, roll, pitch, etc.)
• A 3D Pr array [depth samples × pings × transceivers] of received power
• Optionally a 4D Ang array [depth samples × pings × transceivers × 2] of electrical angles (alongship, athwartship)
• Calls: read.EK_raw → get_dgIdx → dgTime

parse.nmea

Parses NMEA text strings from get_NME0 output. Automatically selects the best GPS sentence type (GPRMC → GPGGA → INGGA → GPGLL). Extracts GPS time, longitude, latitude, and optionally bearing and speed from VTG sentences.

---

Layer 3 — File Readers (produce data objects)

read.EK60_raw

Main entry point for EK60 RAW files. Orchestrates the full import pipeline:

1. read.EK_raw → raw bytes
2. get_CON0 → configuration
3. get_NME0 → NMEA sentences (optionally parsed via parse.nmea)
4. get_RAW0 → acoustic sample data + power + angles

Returns a list: $configuration, $nmea, $pings.

read.EK_bot

Reads Simrad BOT files (bottom detection). Extracts ping times and detected bottom depths for each transceiver from BOT0 datagrams.

• Calls: read.EK_raw → get_dgIdx → dgTime

read.EK_idx

Reads Simrad IDX files (ping index). Extracts ping number, vessel distance, latitude, longitude, and file offset from IDX0 datagrams.

• Calls: read.EK_raw → get_dgIdx → dgTime

read.echogram

Main entry point for HAC files. Imports data from an HAC file using the readHAC package:

1. Identifies available tuple types (echosounder, channel, ping)
2. Extracts Sv matrix from ping tuples, handling varying matrix sizes via mergeSvmat
3. Calculates sample depth vector from time sample interval and sound speed
4. Calls bottom.hac, position.hac, navigation.hac for per-ping metadata
5. Matches navigation speed to pings and computes cumulative distance

Returns an object of class "echogram".

ek2echogram

Converts imported EK60 data to class "echogram". Bridge between the RAW pipeline and the echogram analysis functions:

1. Calls sampleRange for depth vector
2. Calls convertPower for Sv (or TS) matrix
3. Assembles the result with ping times

Accepts optional environment (sound velocity, absorption) and calibration (gain, Sa correction) overrides.

sampleRange

Calculates the depth vector from EK60 data using sound speed and sample interval: $R = \frac{c \cdot \Delta t}{2}$ (cumulative).

convertPower

Converts received power (Pr) to Sv or TS using the sonar equation:

$$S_v(R) = P_r + 20\log_{10}(R) + 2\alpha R - 10\log_{10}!\left(\frac{P_t G_0^2 \lambda^2}{16\pi^2}\right) - 10\log_{10}!\left(\frac{c,\tau,\psi}{2}\right) - 2,S_a\text{corr}$$

Uses transceiver parameters (gain, frequency, EBA) and per-ping metadata (power, pulse length, sound speed, absorption).

convertAngles

Converts electrical angles (alongship/athwartship) to mechanical angles using transceiver sensitivity and offset parameters.

---

HAC Helper Functions

position.hac

Imports GPS positions from the Position tuple (type 20) in an HAC file: GPS time, CPU time, longitude, latitude.

navigation.hac

Computes bearing (rhumb-line), distance (Vincenty ellipsoid), time difference, and speed between consecutive GPS fixes. Uses the geosphere package.

bottom.hac

Extracts detected bottom range and ping times from the ping tuple in an HAC file. Optionally produces a plot.

mergeSvmat

Merges two Sv matrices with unequal row counts by padding the shorter one with NA rows, then column-binding. Used internally by read.echogram and join.echogram.

---

Layer 4 — Visualisation & Analysis

echogram

The primary plotting function. Produces a colour-coded echogram image from an "echogram" object:

• Configurable Sv threshold (Svthr), max (Svmax), and colour separation (col.sep)
• Multiple colour schemes: "parula" (default, via pals), "EK500", "echov", custom vectors, or functions
• X-axis reference: ping number, nautical miles, or time
• Optional detected bottom line overlay
• Optional colour bar (uses an embedded imageScale function from sinkr)
• Calls trim.echogram internally when depth.max / ping.ini / ping.max are specified

palette.echogram

Designs colour palettes for echograms. Can use three built-in schemes or custom colour vectors, interpolating via colorRampPalette. Returns a list of palette (colours) and breaks (Sv boundaries). Optionally displays a visual preview.

trim.echogram

Crops an echogram vertically (by maximum depth) and/or horizontally (by ping range) by subsetting the Sv matrix, depth vector, and pings data.frame.

join.echogram

Merges two echograms of the same frequency end-to-end (column-wise). Checks that frequencies match, uses mergeSvmat to handle different depth ranges, and recalculates cumulative distance.

match.echogram

Synchronises two echograms from different frequencies by matching ping times. Removes non-matching and duplicated pings so that both echograms have identical dimensions — a prerequisite for add.echogram.

add.echogram

Adds or subtracts Sv matrices from two frequencies, in either the dB or linear domain. This is the core operation for multi-frequency acoustic filtering. Requires matched echograms (use match.echogram first) and masked echograms (use mask.echogram first to remove bottom echoes).

mask.echogram

Creates and applies a mask to blank out unwanted regions: surface layer (by surf.off offset in metres) and/or below-bottom region (by bott.off offset relative to detected bottom). The mask is a matrix of 1's and NA's.

sample.echogram

Interactive pixel sampling — click on an echogram to extract Sv values, ping times, and depths at specific locations. Supports coordinate transfer between frequencies (sample points on one frequency, then retrieve corresponding values from another).

noise.echogram

Models background ambient noise using the equation from De Robertis & Higginbottom (2007):

$$N(R) = N_{1m} + 20\log_{10}(R) + 2\alpha(R - 1)$$

Fits the noise level at 1 m depth (dB1m) and absorption coefficient (alpha) to observed Sv data. Can output a noise echogram for subsequent subtraction.

---

Typical Workflows

Workflow 1: HAC file → Echogram

# Import
hacfile <- system.file("extdata", "D20150510-T202500.hac", package = "echogram")
echo.038 <- read.echogram(hacfile, channel = 1)

# Visualise
echogram(echo.038, Svthr = -70, scheme = "EK500")

Workflow 2: EK60 RAW file → Echogram

# Import raw
ek <- read.EK60_raw("file.raw", parseNMEA = TRUE, angles = TRUE)

# Convert to echogram
eco <- ek2echogram(ek, frequency = 1, data = "Sv")
echogram(eco)

Workflow 3: Multi-frequency filtering (dB difference)

# Import two frequencies
echo.038 <- read.echogram(hacfile, channel = 1)
echo.120 <- read.echogram(hacfile, channel = 2)

# Synchronise pings
tmp <- match.echogram(echo.038, echo.120)
echo.038 <- tmp$echogram1
echo.120 <- tmp$echogram2

# Mask surface and bottom
echo.038m <- mask.echogram(echo.038, surf.off = 2, bott.off = 0.2)
echo.120m <- mask.echogram(echo.120, surf.off = 2, bott.off = 0.2)

# Subtract: 120 kHz − 38 kHz (positive = krill-like, negative = fish-like)
echo.diff <- add.echogram(echo.120m, echo.038m, "minus", "dB")
echogram(echo.diff, Svthr = floor(min(echo.diff$Sv, na.rm = TRUE)))

Workflow 4: Noise estimation and removal

data("echo.noise")
noise <- noise.echogram(echo.noise, ping = 2, dB1m = -131, alpha = 0.03550554, out = TRUE)
# subtract noise from real echogram using add.echogram(..., "minus", "linear")

---

Dependency Graph (R packages)

Package	Purpose
readHAC	Parse HAC file format (tuples)
geosphere	Geodesic distance and rhumb-line bearing calculations
pals	Default parula colour palette
data.table	rbindlist for transceiver config assembly
plyr	ldply for NMEA parsing

---

Included Data

File	Description
data/echo.noise.rda	Example echogram (120 kHz, deep water, no scatterers) for noise estimation
inst/extdata/*.hac	Demo HAC files at 38 and 120 kHz
inst/extdata/*.bot	Demo bottom-detection file
inst/extdata/*.idx	Demo ping-index file