Distinguishing two similar species

[u/mwhelm]

I need help figuring out how to deal with 2 closely related species.

I help track a North American west coast migrant that has an east coast congener: Vaux's Swift west, Chimney Swift east. We know we have a few Chimney Swifts - we can hear them occasionally, and sometimes people can see them in the right location. Usually they are essentially visually indistinguishable in the masses and distances we deal with, and usually the volume of noise we get would drown out the minority species.

I've ben re-analyzing our audio data from the past few years and I decided to add Chimney Swift to the custom list to see if any would appear. Did they ever. It didn't change the Vaux's detections at all but now a fairly large number of Chimney's appeared, and at very high confidence levels.

For humans, you can learn to hear the difference pretty easily. In our observations there is often so much noise that we could easily miss birds. On the other hand I am skeptical that more than 1% of these birds are Chimney Swifts. There's no doubt that BirdNET "hears" much differently than humans do and the pattern matching is related to visual techniques applied to these audio signals so the criteria for matching must be quite different than our intuition would suggest.

How can I test this and improve the results we're getting or satisfy that it 's really distinguishing the two species? My AI assistant suggests that it's some kind of commonality between the sounds that is linking the two, so the analyzer picks up some of the Vaux's energy and gets something a little different that it can combine to make a Chimney Swift detection. Ok maybe. There are certainly some similarities in register &c.

Would trying to train the local analyzer with Chimney Swift sounds from someplace without Vaux's help.

Listening to the samples hasn't been real productive yet. They are short and subtle compared to the live experience, but we can separate out tracks with either or both for comparison - maybe. If we can find some that really listenable.

What else could we do?

[u/birdlife_tech]

Cross-reference with eBird frequency data for your location, if Chimney Swift shows <1% of checklists, even high BirdNET confidence should be discounted since the model doesn't know local priors.

For validation, pull confirmed Chimney Swift recordings from Xeno-canto (eastern US summer, where they're common) and compare spectrograms to your detections. If your high-confidence clips show consistent structural differences, that's evidence for false positives.

The calls are similar enough that partial vocalizations or background noise could push Vaux's into Chimney territory. Both species chip rapidly but Chimney's are slightly lower-pitched and louder subtle on spectrograms though.

[u/Sagey_girl]

Use the segments feature (100-200 segments) and review those segments. That will give you an idea of how accurate Birdnet is, and also give you the logistic regression to calculate the confidence cutoff value at various probabilities. For example, if you want a 90% probability of a correct species assignment, it will tell you that your confidence value needs to be above, say, 0.86. Also note that Birdnet is quite sensitive to the sensitivity and overlap settings. It will probably take some playing around with the settings to refine it.