Returns layout transformation matrix that applies an optional mirror effect and compensates for video vs display aspect ratio.