Field Notes Journal Entry
When the Clusters Blur
Exploring hierarchical seasonal structure, ecological neighbourhoods, and the problem of boundaries within the species similarity system
Yesterday’s heatmap raised an obvious next question.
If visible seasonal neighbourhoods were genuinely emerging from the similarity system, could those neighbourhoods be extracted explicitly?
At first, this seemed fairly straightforward. The similarity matrix already contained a hierarchical clustering structure internally, and the heatmap itself was generated by arranging species according to that hierarchy. In principle, extracting clusters simply meant cutting the hierarchy into groups.
In practice, however, the process turned out to be more interesting than expected.
From Neighbourhoods to Clusters
The heatmap showed several fairly clear regions along the diagonal and, initially, it was tempting to treat these regions as clean ecological partitions.
However, once explicit cluster extraction was added, something slightly different appeared - some neighbourhoods remained extremely coherent, while others began to blur at the edges.
In one case, a large final cluster appeared visually to contain at least two distinct neighbourhoods despite technically belonging to the same higher-level cluster.
At first glance, this looked like a possible inconsistency between the heatmap and the clustering system.
In reality, it turned out to reflect something more important: The structure itself is hierarchical.
Hierarchy Rather Than Hard Boundaries
The heatmap visualises continuous similarity structure. The extracted clusters, by contrast, require hard decisions - at some point, the hierarchy must be cut into discrete groups, and that distinction matters.
A species may sit comfortably within a broad “resident bird” assemblage while still belonging visually to a smaller and more distinct local neighbourhood inside that larger structure.
For example, one part of the heatmap may contain:
- Blackbird
- Robin
- Song Thrush
While a nearby sub-region contains:
- Blue Tit
- Great Tit
- Dunnock
Both regions may still belong to the same higher-level resident cluster while clearly forming recognisable sub-assemblages internally.
The resulting structure feels substantially more ecological than a set of perfectly clean partitions would have done.
Real ecological systems often contain:
- Gradients
- Transitional species
- Overlapping seasonal behaviour
- Nested assemblages
- Fuzzy boundaries
The clustering system now appears to be recovering at least some of that structure naturally.
Aligning the Visual and Analytical Structure
One useful refinement during the process was ensuring that both the heatmap and the extracted clusters were generated from the same underlying hierarchical tree.
Earlier versions produced slightly different interpretations because the clustering logic was duplicated independently between the visualisation and analysis stages. Refactoring this into a shared clustering framework made the resulting structure substantially easier to interpret.
The heatmap ordering, dendrogram hierarchy, and extracted clusters now all represent different views of the same underlying seasonal similarity structure.
That consistency matters because it becomes much easier to understand where the boundaries genuinely blur and where distinct neighbourhoods actually exist.
The Problem of Resolution
Another interesting outcome was that the apparent structure changes depending on the resolution at which the hierarchy is cut.
Broad cluster counts produce large ecological regions:
- Residents
- Spring seasonal species
- Winter visitors
Finer cuts reveal progressively smaller neighbourhoods and transitional assemblages.
Neither view is necessarily “correct”. They simply represent different levels within the same hierarchy.
This feels increasingly similar to ecological observation itself as the natural world rarely divides cleanly into perfectly isolated compartments.
Instead, patterns emerge at multiple overlapping scales simultaneously.
What the Clusters May Represent
At present, the clusters remain exploratory rather than definitive and they should not be interpreted as formal ecological guilds or objective biological categories.
The current system is still driven by:
- Feature extraction choices
- Similarity weighting
- Clustering parameters
- Relatively small datasets
Nevertheless, some of the resulting assemblages appear surprisingly plausible.
Perhaps most interestingly, the strongest relationships are not always taxonomic. Butterflies may cluster near flowering periods. Migratory birds may align with broader spring transition structure. Seasonal proximity appears capable of crossing traditional biological categories entirely.
One extracted neighbourhood, for example, grouped together:
- Bluebell
- Garlic Mustard
- Cow Parsley
- Cowslip
- Orange Tip Butterfly
All five species share a strong spring seasonal signal, despite spanning both flowering plants and insects.
The resulting assemblage appears less like a taxonomic grouping and more like a small fragment of the spring ecological transition itself.
This suggests the system may be recovering aspects of phenological organisation rather than simple species similarity.
A More Interesting Kind of Ambiguity
One of the more encouraging aspects of the current results is that the structure is not perfectly tidy. If every species fell into completely clean partitions, the ecology would likely feel artificially simplified.
Instead, the hierarchy appears to contain:
- Strong neighbourhoods
- Transitional bridges
- Nested structure
- Overlapping assemblages
- Ambiguous edge cases
Those ambiguities may ultimately prove more informative than the cleanest clusters themselves.
For now, at least, the system appears to be moving away from individual species modelling and towards something broader: An exploratory map of seasonal ecological structure.
Tool
ODE Solver
A simple tool for exploring time-based models
The seasonal presence and detectability models were developed using a small, general-purpose ordinary differential equation solver, designed for experimentation and visualisation.
It allows simple systems to be defined and explored over time, making it possible to test how patterns might arise from underlying processes.
The application, the models, and instructions on how to run them are provided in the GitHub repository.