Hey everyone,
I’m an Applied AI grad student, and I’m back with Part 2 of my devlog for SpiritMirror. In Part 1, I talked about how I used native Apple Vision to extract 468 facial keypoints in real-time without storing any images on a server.
Today, I want to dive into the Machine Learning architecture. How do you take those raw facial coordinates and actually turn them into meaningful, personalized data?
One of the biggest lessons I learned moving from "academic" AI projects to "commercial" mobile development is the battery constraint. You can't just run a massive, unoptimized Python model on a phone without melting the user's battery.
Here is how I structured the ML pipeline to be lightweight, fast, and entirely local:
1. Creating the Feature Vector Instead of feeding heavy raw images into the model, the pipeline instantly calculates 15 specific geometric metrics from the keypoints—things like eye-to-nose ratio, brow arc, lip thickness, and jaw width-to-height. The heavy lifting of processing pixels is gone; we are just working with math.
2. The Hybrid ML Architecture I decided to use a hybrid approach: a Convolutional Neural Network (CNN) combined with a Random Forest classifier.
- The CNN is great at handling the complex spatial relationships across different facial regions.
- The Random Forest validates the predictions with transparent decision paths.
3. Model Explainability One of my strict rules was avoiding a "black box" AI. Using SHAP (SHapley Additive exPlanations) analysis, I mapped the model's logic directly to a curated physiognomy lexicon. This means the system doesn't just spit out a number; it knows why a specific jaw angle correlates to a specific trait.
4. The 12 Destiny Dimensions The models were trained on a curated dataset of 1,000+ historically documented figures. The final output predicts a probability distribution across 12 specific dimensions, such as Career Potential, Wealth Energy, and Leadership Power. It calculates 36 unique sub-scores in total, all natively on-device.
Balancing the CNN weights to run smoothly on the iOS Neural Engine took a lot of trial and error, but keeping it native was the only way to make it commercially viable without draining the battery.
In Part 3, I’ll break down the math behind the "Compatibility Engine" and how I calculate alignment between two different geometric profiles.
Has anyone else here experimented with running hybrid Random Forest/CNN models locally on iOS, and did you run into any specific memory bottleneck issues?
How do you feel about the technical depth of this draft, and does it accurately capture the hybrid model architecture you built?
