• zaidenz

How to Dynamically change color of AR Material in Xcode.(Face Color)

To start, download this starter code:

Facial recognition in Xcode is made up of three main components. Your AR Tracking Configuration, your Render Functions, and your Expressions.

First thing you want to do is navigate to your renderer functions, and change your fill mode from lines to fill.

node.geometry?.firstMaterial?.fillMode = .fill

when you run the program, it should look like this:

To add some depth, the next thing you want to change is the lighting model of your material. You can do this by accessing the geometry?.firstMaterial?.lightingModel. I made a variable instead, both ways would work.

let material = node.geometry?.firstMaterial
material?.lightingModel = .physicallyBased

By changing the lighting to "physically based" you are telling Xcode you want the material to simulate light hitting the face mesh. When you run this it should look something like this:

It is a little creepy I know...

Now we want to change colors dynamically. Luckily for us we have some base functionality built in to our starter code. We can use our expressions as triggers!

But first, how do we access the color property? For this we need to think in terms of 3D modelling and texturing terminology. When creating a material for an object, we need to access its diffuse property to change the color.


But for it to be dynamic we do not want to hardcode a value, so at the top of your code create a new UIColor property and assign it with some RBG values.

 @IBOutlet var sceneView: ARSCNView!
 @IBOutlet var label: UILabel!
 var action = ""
 var color = UIColor(red: 1.0, green: 1.0, blue: 1.0, alpha: 1.0)

Now we can use this to set our initial render color, AND update via expression:

 func renderer(_ renderer: SCNSceneRenderer, nodeFor anchor: ARAnchor) -> SCNNode? {
 let faceMesh = ARSCNFaceGeometry(device: sceneView.device!)
 let node = SCNNode(geometry: faceMesh)
 //set to fill
        node.geometry?.firstMaterial?.fillMode = .fill
 //example of changing color.
       node.geometry?.firstMaterial?.diffuse.contents = color
 //fill face with a material
 let material = node.geometry?.firstMaterial
        material?.lightingModel = .physicallyBased
 return node

func renderer(_ renderer: SCNSceneRenderer, didUpdate node: SCNNode, for anchor: ARAnchor) {
 if let faceAnchor = anchor as? ARFaceAnchor, let faceGeometry = node.geometry as? ARSCNFaceGeometry {
                faceGeometry.update(from: faceAnchor.geometry)
 expression(anchor: faceAnchor)
 node.geometry?.firstMaterial?.diffuse.contents = color
 DispatchQueue.main.async {
 self.label.text = self.action

And an expression example:

if jawLeft?.decimalValue ?? 0.0 > 0.1 {
 self.action = "yellow"
 color = UIColor(red: 1.0, green: 1.0, blue: 0.0, alpha: 1.0);

add some funky colors and this should be your result:

Thanks for reading!

8 views0 comments

Recent Posts

See All

Native Development vs Web Development....FIGHT

But in reality, it's not really a fight. Let's talk about the pros and cons of developing applications using Native codebases or Web Development. Native Development Pros: - Can develop applications us