Fully implemented mirror of YouTube's YTApiaryDeviceCrypto class

README.md

LMApiaryDeviceCrypto

I was interested in what would go into writing my own lightweight YouTube Music client. This was my step one.

Steps to Step 1

With any client, there's a server. To find out how I could write a client, I needed to find out how Google's client communicated with the server. After inspecting a the HTTP traffic, I came to the conclusion there were four things I would need:

0. API key

1. Authorization HTTP header field

2. X-Goog-Device-Auth HTTP header field

3. X-Goog-Visitor-Id HTTP header field

The API key was easy to get. YouTube includes it in the standard GoogleService-Info.plist (AIzaSyC4SSoMBxVCNqJJEIuxYZa5WVFqZUurXjc), and YouTube Music included a few in the binary. A quick strings and grep turned up five keys:

AIzaSyBmltRCNALB9rnWNIiy5FUd-LpDVvYYbGE
AIzaSyAdz24CjJsPc74_a3hV9nxIJSburJjtJe8
AIzaSyDK3iBpDP9nHVTk2qL73FLJICfOC3c51Og
AIzaSyA5iahgfY6tRS9bOBoWAcosfzDu69-juHo
AIzaSyDDAbQV7WXHydhJm-qArV2aCSKs1reexhk

I found this interesting. As far as I could tell, two of these were used in the app. Some of the ones I tested seemed to be registered in a database somewhere, but not fully setup: YouTube Internal API (InnerTube) has not been used in project 75882956776 before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/youtubei.googleapis.com/overview?project=75882956776 then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.

With this out of the way, I moved on to the Authorization HTTP header field. It seemed like this was hardcoded as well, however I later found out that was incorrect. Since I didn't know that, I moved onto the device X-Goog-Device-Auth HTTP header field.

Reverse Engineering

YTApiaryDeviceCrypto is responsible for "signing" NSMutableURLRequests in YouTube clients on iOS. To be able to make my own requests, I would need the functionality of this class too. I decided the best way to do that was reimplement the class. Using Hopper, cycript, and MobileSubstrate, I was able to do this.

I ran many tests, and after I was confident my implementation was the same as Google's, I wanted to share, in the hopes that others find it useful. I documented the header to my best abilities with helpful, relevant information.

LMApiaryDeviceCrypto.h

//
//  LMApiaryDeviceCrypto.h
//
//  Created by Leptos on 11/18/18.
//  Copyright © 2018 Leptos. All rights reserved.
//

#import <Foundation/Foundation.h>

#define kYouTubeBase64EncodedProjectKey @"vOU14u6GkupSL2pLKI/B7L3pBZJpI8W92RoKHJOu3PY="
#define kYouTubeMusicBase64EncodedProjectKey @"WrM95onSB5FfXofSzKWgkNZGiosfmCCAcTH4htvkuj4="

/// Fully implemented mirror of @c YTApiaryDeviceCrypto.
/// Some implementations have been slightly edited,
/// however anything created with either class will work with the other,
/// including Archives, however this is SecureCoding, and YT is not.
/// An instance of this class should be created once, and stored to disk using @c NSKeyedArchiver.
/// This class is able to sign @c NSMutableURLRequest objects for interaction with private YouTube REST API.
@interface LMApiaryDeviceCrypto : NSObject <NSSecureCoding>

/// For convenience, the Base64 encoded values of the project keys for YouTube and YouTube Music are included above.
/// @param hmacLength Specify 4
- (instancetype)initWithProjectKey:(NSData *)projectKey HMACLength:(NSUInteger)hmacLength;
/// Before signing a URL request, the deviceID and deviceKey must be set.
/// A valid deviceID and deviceKey can be obtained by making an HTTP POST to
/// https://youtubei.googleapis.com/deviceregistration/v1/devices?key=API_KEY&rawDeviceId=RAW_DEVICE_ID
/// where API_KEY is a valid API key, and RAW_DEVICE_ID is any nonnull string (may support only UUIDs in the future)
- (BOOL)setDeviceID:(NSString *)deviceID deviceKey:(NSString *)deviceKey error:(NSError **)error;

/// Signing a URL request uses all the compotents of this class- ensure they are all valid before calling.
/// The signature is placed in the @c X-Goog-Device-Auth HTTP header field.
/// Mutating the URL or HTTPBody of the @c request after signing it is invalid- other header fields are safe
- (BOOL)signURLRequest:(NSMutableURLRequest *)request error:(NSError **)error;

/// Decrypt an encoded string that was encrypted with the same mechanism, and project key.
- (NSData *)decryptEncodedString:(NSString *)encodedString error:(NSError **)error;
/// Ecrypt and encode any given data. The result is unlikey to be the same for any given data,
/// however the input to this method will always be equal to the output of @c decryptEncodedString:error:
/// when the output of this method is passed as the input to the decrypt method.
/// @code
/// LMApiaryDeviceCrypto *deviceCrypto = ...;
/// NSData *startData = ...;
/// NSString *encryptAlpha = [deviceCrypto encryptAndEncodeData:startData error:NULL];
/// NSString *encryptBeta  = [deviceCrypto encryptAndEncodeData:startData error:NULL];
/// [encryptAlpha isEqualToString:encryptBeta]; // Chance of being YES is 1 << 64
/// NSData *endDataAlpha = [deviceCrypto decryptEncodedString:encryptAlpha error:NULL];
/// NSData *endDataBeta  = [deviceCrypto decryptEncodedString:encryptBeta  error:NULL];
/// [endDataAlpha isEqualToData:endDataBeta]; // Should always be YES
/// [endDataAlpha isEqualToData:startData];   // Should always be YES
/// @endcode
- (NSString *)encryptAndEncodeData:(NSData *)data error:(NSError **)error;

@end

LMApiaryDeviceCrypto.m

//
//  LMApiaryDeviceCrypto.m
//
//  Created by Leptos on 11/18/18.
//  Copyright © 2018 Leptos. All rights reserved.
//

#import <CommonCrypto/CommonCrypto.h>

#import "LMApiaryDeviceCrypto.h"
#import "../GoogleOSS/GTMStringEncoding.h" /* https://github.com/google/google-toolbox-for-mac/blob/master/Foundation/GTMStringEncoding.h */

#define kYTApiaryDeviceCryptoDeviceIdKey  @"kYTApiaryDeviceCryptoDeviceIdKey"
#define kYTApiaryDeviceCryptoDeviceKeyKey @"kYTApiaryDeviceCryptoDeviceKeyKey"
#define kYTDeviceCryptoProjectKeyKey      @"kYTDeviceCryptoProjectKeyKey"
#define kYTDeviceCryptoHMACKeyKey         @"kYTDeviceCryptoHMACKeyKey"
#define kYTDeviceCryptoHMACLengthKey      @"kYTDeviceCryptoHMACLengthKey"

@interface NSError (LMNetCryptoError)
+ (instancetype)netCryptoErrorWithMessage:(NSString *)message;
@end

@implementation LMApiaryDeviceCrypto {
    NSString *_deviceID;
    NSData *_deviceKey;
    NSData *_projectKey;
    NSData *_hmacKey;
    NSUInteger _hmacLength;
}

- (instancetype)init {
    /* original implementation calls [self class] first? */
    return nil;
}

- (instancetype)initWithProjectKey:(NSData *)projectKey HMACLength:(NSUInteger)hmacLength {
    if (self = [super init]) {
        _hmacLength = hmacLength;
        NSUInteger internalHmacLength = 0x10;
        NSUInteger projectKeyLength = projectKey.length;
        if (projectKeyLength >= internalHmacLength) {
            _projectKey = [projectKey subdataWithRange:NSMakeRange(0, internalHmacLength)];
            _hmacKey = [projectKey subdataWithRange:NSMakeRange(internalHmacLength, projectKeyLength-internalHmacLength)];
        }
    }
    return self;
}

- (BOOL)setDeviceID:(NSString *)deviceID deviceKey:(NSString *)deviceKey error:(NSError **)error {
    NSError *derefError = nil;
    _deviceKey = [self decryptEncodedString:deviceKey error:&derefError];
    if (derefError) {
        if (error) {
            *error = derefError;
        }
        return NO;
    } else {
        _deviceID = [deviceID copy];
        return YES;
    }
}

- (BOOL)signURLRequest:(NSMutableURLRequest *)request error:(NSError **)error {
    NSData *urlData = [request.URL.absoluteString dataUsingEncoding:NSUTF8StringEncoding];
    
    NSString *signedURL = [self signData:urlData padData:YES HMACLength:4];
    NSString *signedContent = [self signData:request.HTTPBody padData:NO HMACLength:CC_SHA1_DIGEST_LENGTH];
    
    NSString *compoundValue = [NSString stringWithFormat:@"device_id=%@,data=%@,content=%@", _deviceID, signedURL, signedContent];
    [request setValue:compoundValue forHTTPHeaderField:@"X-Goog-Device-Auth"];
    
    return YES;
}

- (NSString *)signData:(NSData *)data padData:(BOOL)shouldPad HMACLength:(NSUInteger)hmacLength {
    uint8_t sha1Digest[CC_SHA1_DIGEST_LENGTH];
    CC_SHA1(_deviceKey.bytes, (CC_LONG)_deviceKey.length, sha1Digest);
    NSData *hashedData = [NSData dataWithBytes:sha1Digest length:4];
    
    if (shouldPad) {
        NSUInteger padDataCapacity = data.length + 1;
        NSMutableData *padData = [NSMutableData dataWithCapacity:padDataCapacity];
        [padData appendData:data];
        padData.length = padDataCapacity;
        data = [padData copy];
    }
    CCHmac(kCCHmacAlgSHA1, _deviceKey.bytes, (size_t)_deviceKey.length, data.bytes, data.length, sha1Digest);
    NSMutableData *newData = [NSMutableData data];
    
    uint8_t zeroByte = 0;
    [newData appendBytes:&zeroByte length:sizeof(zeroByte)];
    [newData appendData:hashedData];
    size_t appendLength = sizeof(sha1Digest);
    if (hmacLength < appendLength) {
        appendLength = hmacLength;
    }
    [newData appendBytes:sha1Digest length:appendLength];
    
    GTMStringEncoding *stringEncoder = [GTMStringEncoding rfc4648Base64StringEncoding];
    stringEncoder.doPad = NO;
    return [stringEncoder encode:newData error:NULL];
}

- (NSData *)performCrypto:(NSData *)data outputLength:(NSUInteger)length IV:(NSData *)iv operation:(CCOperation)op {
    CCCryptorRef cryptor;
    CCCryptorStatus status = CCCryptorCreateWithMode(op, kCCModeCTR, kCCAlgorithmAES, ccNoPadding,
                                                     iv.bytes, _projectKey.bytes, 0x10, NULL, 0, 0, kCCModeOptionCTR_BE, &cryptor);
    if (status == kCCSuccess) {
        size_t cryptorLen = CCCryptorGetOutputLength(cryptor, data.length, true);
        NSMutableData *ret = [NSMutableData dataWithLength:cryptorLen];
        size_t dataMoved;
        status = CCCryptorUpdate(cryptor, data.bytes, data.length, ret.mutableBytes, cryptorLen, &dataMoved);
        if (status == kCCSuccess) {
            CCCryptorRelease(cryptor);
            ret.length = length;
            return [ret copy];
        }
    }
    return nil;
}

- (NSData *)paddedData:(NSData *)data {
    NSUInteger dataLength = data.length;
    NSUInteger lengthMod = dataLength & 0xf;
    if (lengthMod) {
        NSMutableData *padData = [NSMutableData dataWithLength:dataLength + 0x10 - lengthMod];
        [padData replaceBytesInRange:NSMakeRange(0, dataLength) withBytes:data.bytes];
        return [padData copy]; /* copy call not in original */
    } else {
        return data;
    }
}

- (NSData *)projectKeySignature {
    NSMutableData *data = [NSMutableData dataWithCapacity:_hmacKey.length + 0x20];
    uint64_t magic = 0x1000000000000000;
    [data appendBytes:&magic length:sizeof(magic)];
    [data appendData:_projectKey];
    [data appendBytes:&magic length:sizeof(magic)];
    [data appendData:_hmacKey];
    
    uint8_t sha1Digest[CC_SHA1_DIGEST_LENGTH];
    CC_SHA1(data.bytes, (CC_LONG)data.length, sha1Digest);
    
    return [NSData dataWithBytes:sha1Digest length:4];
}

- (NSData *)decryptEncodedString:(NSString *)encodedString error:(NSError **)error {
    GTMStringEncoding *strEnc = [GTMStringEncoding rfc4648Base64StringEncoding];
    NSData *decoded = [strEnc decode:encodedString error:error];
    uint8_t firstByte;
    [decoded getBytes:&firstByte length:sizeof(firstByte)];
    if (firstByte == 0) {
        if (decoded.length > 0xc) {
            NSData *lowPad = [self paddedData:[decoded subdataWithRange:NSMakeRange(5, 8)]];
            NSInteger someVal = decoded.length - _hmacLength - 0xd;
            if (someVal >= 0) {
                if ([self verifySignedData:decoded]) {
                    NSData *highPad = [self paddedData:[decoded subdataWithRange:NSMakeRange(0xd, someVal)]];
                    return [self performCrypto:highPad outputLength:someVal IV:lowPad operation:kCCDecrypt];
                } else if (error) {
                    *error = [NSError netCryptoErrorWithMessage:@"Could not verify encrypted data"];
                }
            } else if (error) {
                *error = [NSError netCryptoErrorWithMessage:@"Could not determine cipher"];
            }
        } else if (error) {
            *error = [NSError netCryptoErrorWithMessage:@"Could not determine initializion vector"];
        }
    } else if (error) {
        *error = [NSError netCryptoErrorWithMessage:@"Could not determine key sign"];
    }
    return nil;
}

- (NSString *)encryptAndEncodeData:(NSData *)data error:(NSError **)error {
    NSMutableData *mutData = [NSMutableData data];
    int8_t zeroByte = 0;
    [mutData appendBytes:&zeroByte length:sizeof(zeroByte)];
    [mutData appendData:[self projectKeySignature]];
    
    uint8_t buff[8]; /* however you want this to be 8 bytes; could use a single uint64_t */
    arc4random_buf(buff, sizeof(buff));
    NSData *ivData = [NSData dataWithBytes:buff length:sizeof(buff)];
    [mutData appendData:ivData];
    
    NSData *crypto = [self performCrypto:[self paddedData:data] outputLength:data.length IV:[self paddedData:ivData] operation:kCCEncrypt];
    if (crypto) {
        [mutData appendData:crypto];
        NSMutableData *moreData = [NSMutableData dataWithLength:mutData.length + 9];
        
        uint8_t magicByte = 83;
        [moreData replaceBytesInRange:NSMakeRange(0, 1) withBytes:&magicByte];
        [moreData replaceBytesInRange:NSMakeRange(9, mutData.length) withBytes:mutData.bytes];
        
        uint8_t sha1Digest[CC_SHA1_DIGEST_LENGTH];
        CCHmac(kCCHmacAlgSHA1, _hmacKey.bytes, (size_t)_hmacKey.length, moreData.bytes, moreData.length, sha1Digest);
        [mutData appendBytes:sha1Digest length:_hmacLength];
        
        GTMStringEncoding *strEncode = [GTMStringEncoding rfc4648Base64StringEncoding];
        return [strEncode encode:mutData error:error];
    } else if (error) {
        *error = [NSError netCryptoErrorWithMessage:@"Generic crypto error."];
    }
    return nil;
}

- (BOOL)verifySignedData:(NSData *)data {
    NSData *projectHash = [data subdataWithRange:NSMakeRange(1, 4)];
    if ([projectHash isEqualToData:[self projectKeySignature]]) {
        NSInteger lengthDiff = data.length - _hmacLength;
        if (lengthDiff >= 0) {
            NSData *highData = [data subdataWithRange:NSMakeRange(lengthDiff, _hmacLength)];
            NSData *lowData = [data subdataWithRange:NSMakeRange(0, lengthDiff)];
            NSMutableData *mutData = [NSMutableData dataWithLength:lengthDiff + 9];
            
            uint8_t magicByte = 83;
            [mutData replaceBytesInRange:NSMakeRange(0, 1) withBytes:&magicByte];
            [mutData replaceBytesInRange:NSMakeRange(9, lengthDiff) withBytes:lowData.bytes];
            
            uint8_t hmacBytes[CC_SHA1_DIGEST_LENGTH];
            CCHmac(kCCHmacAlgSHA1, _hmacKey.bytes, _hmacKey.length, mutData.bytes, mutData.length, hmacBytes);
            NSData *checkData = [NSData dataWithBytes:hmacBytes length:_hmacLength];
            return [highData isEqualToData:checkData];
        }
    }
    return NO;
}

// MARK: - Coding

- (instancetype)initWithCoder:(NSCoder *)aDecoder {
    if (self = [super init]) {
        _deviceID   = [aDecoder decodeObjectForKey:kYTApiaryDeviceCryptoDeviceIdKey];
        _deviceKey  = [aDecoder decodeObjectForKey:kYTApiaryDeviceCryptoDeviceKeyKey];
        _projectKey = [aDecoder decodeObjectForKey:kYTDeviceCryptoProjectKeyKey];
        _hmacKey    = [aDecoder decodeObjectForKey:kYTDeviceCryptoHMACKeyKey];
        /* Original uses decodeIntForKey, which is not optimal here */
        _hmacLength = [aDecoder decodeIntegerForKey:kYTDeviceCryptoHMACLengthKey];
    }
    return self;
}

- (void)encodeWithCoder:(NSCoder *)aCoder {
    [aCoder encodeObject:_deviceID    forKey:kYTApiaryDeviceCryptoDeviceIdKey];
    [aCoder encodeObject:_deviceKey   forKey:kYTApiaryDeviceCryptoDeviceKeyKey];
    [aCoder encodeObject:_projectKey  forKey:kYTDeviceCryptoProjectKeyKey];
    [aCoder encodeObject:_hmacKey     forKey:kYTDeviceCryptoHMACKeyKey];
    [aCoder encodeInteger:_hmacLength forKey:kYTDeviceCryptoHMACLengthKey];
}

+ (BOOL)supportsSecureCoding {
    return YES;
}

@end

// MARK: -
@implementation NSError (LMNetCryptoError)

+ (instancetype)netCryptoErrorWithMessage:(NSString *)message {
    return [NSError errorWithDomain:@"com.google.ios.youtube.Net.ErrorDomain" code:0 userInfo:@{
        @"message" : message
    }];
}

@end

@leptos-null Thank you for your speedy response, I've created a gist so as not to clutter up yours (https://gist.github.com/tombulled/d313c54a0681fcf0ba6d8092f11411e6). I've had to redact some values to ensure I'm not leaking any information. Please do take a look at it and I'd really appreciate any help you're able to give!

I reversed the last version in android, and it is quite the same. I made it in PHP, if some needs it, i implemented also fetch player request. Feel free to clone my repo