Fully implemented mirror of YouTube's YTApiaryDeviceCrypto class

README.md

LMApiaryDeviceCrypto

I was interested in what would go into writing my own lightweight YouTube Music client. This was my step one.

Steps to Step 1

With any client, there's a server. To find out how I could write a client, I needed to find out how Google's client communicated with the server. After inspecting a the HTTP traffic, I came to the conclusion there were four things I would need:

0. API key

1. Authorization HTTP header field

2. X-Goog-Device-Auth HTTP header field

3. X-Goog-Visitor-Id HTTP header field

The API key was easy to get. YouTube includes it in the standard GoogleService-Info.plist (AIzaSyC4SSoMBxVCNqJJEIuxYZa5WVFqZUurXjc), and YouTube Music included a few in the binary. A quick strings and grep turned up five keys:

AIzaSyBmltRCNALB9rnWNIiy5FUd-LpDVvYYbGE
AIzaSyAdz24CjJsPc74_a3hV9nxIJSburJjtJe8
AIzaSyDK3iBpDP9nHVTk2qL73FLJICfOC3c51Og
AIzaSyA5iahgfY6tRS9bOBoWAcosfzDu69-juHo
AIzaSyDDAbQV7WXHydhJm-qArV2aCSKs1reexhk

I found this interesting. As far as I could tell, two of these were used in the app. Some of the ones I tested seemed to be registered in a database somewhere, but not fully setup: YouTube Internal API (InnerTube) has not been used in project 75882956776 before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/youtubei.googleapis.com/overview?project=75882956776 then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.

With this out of the way, I moved on to the Authorization HTTP header field. It seemed like this was hardcoded as well, however I later found out that was incorrect. Since I didn't know that, I moved onto the device X-Goog-Device-Auth HTTP header field.

Reverse Engineering

YTApiaryDeviceCrypto is responsible for "signing" NSMutableURLRequests in YouTube clients on iOS. To be able to make my own requests, I would need the functionality of this class too. I decided the best way to do that was reimplement the class. Using Hopper, cycript, and MobileSubstrate, I was able to do this.

I ran many tests, and after I was confident my implementation was the same as Google's, I wanted to share, in the hopes that others find it useful. I documented the header to my best abilities with helpful, relevant information.

LMApiaryDeviceCrypto.h

//
//  LMApiaryDeviceCrypto.h
//
//  Created by Leptos on 11/18/18.
//  Copyright © 2018 Leptos. All rights reserved.
//

#import <Foundation/Foundation.h>

#define kYouTubeBase64EncodedProjectKey @"vOU14u6GkupSL2pLKI/B7L3pBZJpI8W92RoKHJOu3PY="
#define kYouTubeMusicBase64EncodedProjectKey @"WrM95onSB5FfXofSzKWgkNZGiosfmCCAcTH4htvkuj4="

/// Fully implemented mirror of @c YTApiaryDeviceCrypto.
/// Some implementations have been slightly edited,
/// however anything created with either class will work with the other,
/// including Archives, however this is SecureCoding, and YT is not.
/// An instance of this class should be created once, and stored to disk using @c NSKeyedArchiver.
/// This class is able to sign @c NSMutableURLRequest objects for interaction with private YouTube REST API.
@interface LMApiaryDeviceCrypto : NSObject <NSSecureCoding>

/// For convenience, the Base64 encoded values of the project keys for YouTube and YouTube Music are included above.
/// @param hmacLength Specify 4
- (instancetype)initWithProjectKey:(NSData *)projectKey HMACLength:(NSUInteger)hmacLength;
/// Before signing a URL request, the deviceID and deviceKey must be set.
/// A valid deviceID and deviceKey can be obtained by making an HTTP POST to
/// https://youtubei.googleapis.com/deviceregistration/v1/devices?key=API_KEY&rawDeviceId=RAW_DEVICE_ID
/// where API_KEY is a valid API key, and RAW_DEVICE_ID is any nonnull string (may support only UUIDs in the future)
- (BOOL)setDeviceID:(NSString *)deviceID deviceKey:(NSString *)deviceKey error:(NSError **)error;

/// Signing a URL request uses all the compotents of this class- ensure they are all valid before calling.
/// The signature is placed in the @c X-Goog-Device-Auth HTTP header field.
/// Mutating the URL or HTTPBody of the @c request after signing it is invalid- other header fields are safe
- (BOOL)signURLRequest:(NSMutableURLRequest *)request error:(NSError **)error;

/// Decrypt an encoded string that was encrypted with the same mechanism, and project key.
- (NSData *)decryptEncodedString:(NSString *)encodedString error:(NSError **)error;
/// Ecrypt and encode any given data. The result is unlikey to be the same for any given data,
/// however the input to this method will always be equal to the output of @c decryptEncodedString:error:
/// when the output of this method is passed as the input to the decrypt method.
/// @code
/// LMApiaryDeviceCrypto *deviceCrypto = ...;
/// NSData *startData = ...;
/// NSString *encryptAlpha = [deviceCrypto encryptAndEncodeData:startData error:NULL];
/// NSString *encryptBeta  = [deviceCrypto encryptAndEncodeData:startData error:NULL];
/// [encryptAlpha isEqualToString:encryptBeta]; // Chance of being YES is 1 << 64
/// NSData *endDataAlpha = [deviceCrypto decryptEncodedString:encryptAlpha error:NULL];
/// NSData *endDataBeta  = [deviceCrypto decryptEncodedString:encryptBeta  error:NULL];
/// [endDataAlpha isEqualToData:endDataBeta]; // Should always be YES
/// [endDataAlpha isEqualToData:startData];   // Should always be YES
/// @endcode
- (NSString *)encryptAndEncodeData:(NSData *)data error:(NSError **)error;

@end

LMApiaryDeviceCrypto.m

//
//  LMApiaryDeviceCrypto.m
//
//  Created by Leptos on 11/18/18.
//  Copyright © 2018 Leptos. All rights reserved.
//

#import <CommonCrypto/CommonCrypto.h>

#import "LMApiaryDeviceCrypto.h"
#import "../GoogleOSS/GTMStringEncoding.h" /* https://github.com/google/google-toolbox-for-mac/blob/master/Foundation/GTMStringEncoding.h */

#define kYTApiaryDeviceCryptoDeviceIdKey  @"kYTApiaryDeviceCryptoDeviceIdKey"
#define kYTApiaryDeviceCryptoDeviceKeyKey @"kYTApiaryDeviceCryptoDeviceKeyKey"
#define kYTDeviceCryptoProjectKeyKey      @"kYTDeviceCryptoProjectKeyKey"
#define kYTDeviceCryptoHMACKeyKey         @"kYTDeviceCryptoHMACKeyKey"
#define kYTDeviceCryptoHMACLengthKey      @"kYTDeviceCryptoHMACLengthKey"

@interface NSError (LMNetCryptoError)
+ (instancetype)netCryptoErrorWithMessage:(NSString *)message;
@end

@implementation LMApiaryDeviceCrypto {
    NSString *_deviceID;
    NSData *_deviceKey;
    NSData *_projectKey;
    NSData *_hmacKey;
    NSUInteger _hmacLength;
}

- (instancetype)init {
    /* original implementation calls [self class] first? */
    return nil;
}

- (instancetype)initWithProjectKey:(NSData *)projectKey HMACLength:(NSUInteger)hmacLength {
    if (self = [super init]) {
        _hmacLength = hmacLength;
        NSUInteger internalHmacLength = 0x10;
        NSUInteger projectKeyLength = projectKey.length;
        if (projectKeyLength >= internalHmacLength) {
            _projectKey = [projectKey subdataWithRange:NSMakeRange(0, internalHmacLength)];
            _hmacKey = [projectKey subdataWithRange:NSMakeRange(internalHmacLength, projectKeyLength-internalHmacLength)];
        }
    }
    return self;
}

- (BOOL)setDeviceID:(NSString *)deviceID deviceKey:(NSString *)deviceKey error:(NSError **)error {
    NSError *derefError = nil;
    _deviceKey = [self decryptEncodedString:deviceKey error:&derefError];
    if (derefError) {
        if (error) {
            *error = derefError;
        }
        return NO;
    } else {
        _deviceID = [deviceID copy];
        return YES;
    }
}

- (BOOL)signURLRequest:(NSMutableURLRequest *)request error:(NSError **)error {
    NSData *urlData = [request.URL.absoluteString dataUsingEncoding:NSUTF8StringEncoding];
    
    NSString *signedURL = [self signData:urlData padData:YES HMACLength:4];
    NSString *signedContent = [self signData:request.HTTPBody padData:NO HMACLength:CC_SHA1_DIGEST_LENGTH];
    
    NSString *compoundValue = [NSString stringWithFormat:@"device_id=%@,data=%@,content=%@", _deviceID, signedURL, signedContent];
    [request setValue:compoundValue forHTTPHeaderField:@"X-Goog-Device-Auth"];
    
    return YES;
}

- (NSString *)signData:(NSData *)data padData:(BOOL)shouldPad HMACLength:(NSUInteger)hmacLength {
    uint8_t sha1Digest[CC_SHA1_DIGEST_LENGTH];
    CC_SHA1(_deviceKey.bytes, (CC_LONG)_deviceKey.length, sha1Digest);
    NSData *hashedData = [NSData dataWithBytes:sha1Digest length:4];
    
    if (shouldPad) {
        NSUInteger padDataCapacity = data.length + 1;
        NSMutableData *padData = [NSMutableData dataWithCapacity:padDataCapacity];
        [padData appendData:data];
        padData.length = padDataCapacity;
        data = [padData copy];
    }
    CCHmac(kCCHmacAlgSHA1, _deviceKey.bytes, (size_t)_deviceKey.length, data.bytes, data.length, sha1Digest);
    NSMutableData *newData = [NSMutableData data];
    
    uint8_t zeroByte = 0;
    [newData appendBytes:&zeroByte length:sizeof(zeroByte)];
    [newData appendData:hashedData];
    size_t appendLength = sizeof(sha1Digest);
    if (hmacLength < appendLength) {
        appendLength = hmacLength;
    }
    [newData appendBytes:sha1Digest length:appendLength];
    
    GTMStringEncoding *stringEncoder = [GTMStringEncoding rfc4648Base64StringEncoding];
    stringEncoder.doPad = NO;
    return [stringEncoder encode:newData error:NULL];
}

- (NSData *)performCrypto:(NSData *)data outputLength:(NSUInteger)length IV:(NSData *)iv operation:(CCOperation)op {
    CCCryptorRef cryptor;
    CCCryptorStatus status = CCCryptorCreateWithMode(op, kCCModeCTR, kCCAlgorithmAES, ccNoPadding,
                                                     iv.bytes, _projectKey.bytes, 0x10, NULL, 0, 0, kCCModeOptionCTR_BE, &cryptor);
    if (status == kCCSuccess) {
        size_t cryptorLen = CCCryptorGetOutputLength(cryptor, data.length, true);
        NSMutableData *ret = [NSMutableData dataWithLength:cryptorLen];
        size_t dataMoved;
        status = CCCryptorUpdate(cryptor, data.bytes, data.length, ret.mutableBytes, cryptorLen, &dataMoved);
        if (status == kCCSuccess) {
            CCCryptorRelease(cryptor);
            ret.length = length;
            return [ret copy];
        }
    }
    return nil;
}

- (NSData *)paddedData:(NSData *)data {
    NSUInteger dataLength = data.length;
    NSUInteger lengthMod = dataLength & 0xf;
    if (lengthMod) {
        NSMutableData *padData = [NSMutableData dataWithLength:dataLength + 0x10 - lengthMod];
        [padData replaceBytesInRange:NSMakeRange(0, dataLength) withBytes:data.bytes];
        return [padData copy]; /* copy call not in original */
    } else {
        return data;
    }
}

- (NSData *)projectKeySignature {
    NSMutableData *data = [NSMutableData dataWithCapacity:_hmacKey.length + 0x20];
    uint64_t magic = 0x1000000000000000;
    [data appendBytes:&magic length:sizeof(magic)];
    [data appendData:_projectKey];
    [data appendBytes:&magic length:sizeof(magic)];
    [data appendData:_hmacKey];
    
    uint8_t sha1Digest[CC_SHA1_DIGEST_LENGTH];
    CC_SHA1(data.bytes, (CC_LONG)data.length, sha1Digest);
    
    return [NSData dataWithBytes:sha1Digest length:4];
}

- (NSData *)decryptEncodedString:(NSString *)encodedString error:(NSError **)error {
    GTMStringEncoding *strEnc = [GTMStringEncoding rfc4648Base64StringEncoding];
    NSData *decoded = [strEnc decode:encodedString error:error];
    uint8_t firstByte;
    [decoded getBytes:&firstByte length:sizeof(firstByte)];
    if (firstByte == 0) {
        if (decoded.length > 0xc) {
            NSData *lowPad = [self paddedData:[decoded subdataWithRange:NSMakeRange(5, 8)]];
            NSInteger someVal = decoded.length - _hmacLength - 0xd;
            if (someVal >= 0) {
                if ([self verifySignedData:decoded]) {
                    NSData *highPad = [self paddedData:[decoded subdataWithRange:NSMakeRange(0xd, someVal)]];
                    return [self performCrypto:highPad outputLength:someVal IV:lowPad operation:kCCDecrypt];
                } else if (error) {
                    *error = [NSError netCryptoErrorWithMessage:@"Could not verify encrypted data"];
                }
            } else if (error) {
                *error = [NSError netCryptoErrorWithMessage:@"Could not determine cipher"];
            }
        } else if (error) {
            *error = [NSError netCryptoErrorWithMessage:@"Could not determine initializion vector"];
        }
    } else if (error) {
        *error = [NSError netCryptoErrorWithMessage:@"Could not determine key sign"];
    }
    return nil;
}

- (NSString *)encryptAndEncodeData:(NSData *)data error:(NSError **)error {
    NSMutableData *mutData = [NSMutableData data];
    int8_t zeroByte = 0;
    [mutData appendBytes:&zeroByte length:sizeof(zeroByte)];
    [mutData appendData:[self projectKeySignature]];
    
    uint8_t buff[8]; /* however you want this to be 8 bytes; could use a single uint64_t */
    arc4random_buf(buff, sizeof(buff));
    NSData *ivData = [NSData dataWithBytes:buff length:sizeof(buff)];
    [mutData appendData:ivData];
    
    NSData *crypto = [self performCrypto:[self paddedData:data] outputLength:data.length IV:[self paddedData:ivData] operation:kCCEncrypt];
    if (crypto) {
        [mutData appendData:crypto];
        NSMutableData *moreData = [NSMutableData dataWithLength:mutData.length + 9];
        
        uint8_t magicByte = 83;
        [moreData replaceBytesInRange:NSMakeRange(0, 1) withBytes:&magicByte];
        [moreData replaceBytesInRange:NSMakeRange(9, mutData.length) withBytes:mutData.bytes];
        
        uint8_t sha1Digest[CC_SHA1_DIGEST_LENGTH];
        CCHmac(kCCHmacAlgSHA1, _hmacKey.bytes, (size_t)_hmacKey.length, moreData.bytes, moreData.length, sha1Digest);
        [mutData appendBytes:sha1Digest length:_hmacLength];
        
        GTMStringEncoding *strEncode = [GTMStringEncoding rfc4648Base64StringEncoding];
        return [strEncode encode:mutData error:error];
    } else if (error) {
        *error = [NSError netCryptoErrorWithMessage:@"Generic crypto error."];
    }
    return nil;
}

- (BOOL)verifySignedData:(NSData *)data {
    NSData *projectHash = [data subdataWithRange:NSMakeRange(1, 4)];
    if ([projectHash isEqualToData:[self projectKeySignature]]) {
        NSInteger lengthDiff = data.length - _hmacLength;
        if (lengthDiff >= 0) {
            NSData *highData = [data subdataWithRange:NSMakeRange(lengthDiff, _hmacLength)];
            NSData *lowData = [data subdataWithRange:NSMakeRange(0, lengthDiff)];
            NSMutableData *mutData = [NSMutableData dataWithLength:lengthDiff + 9];
            
            uint8_t magicByte = 83;
            [mutData replaceBytesInRange:NSMakeRange(0, 1) withBytes:&magicByte];
            [mutData replaceBytesInRange:NSMakeRange(9, lengthDiff) withBytes:lowData.bytes];
            
            uint8_t hmacBytes[CC_SHA1_DIGEST_LENGTH];
            CCHmac(kCCHmacAlgSHA1, _hmacKey.bytes, _hmacKey.length, mutData.bytes, mutData.length, hmacBytes);
            NSData *checkData = [NSData dataWithBytes:hmacBytes length:_hmacLength];
            return [highData isEqualToData:checkData];
        }
    }
    return NO;
}

// MARK: - Coding

- (instancetype)initWithCoder:(NSCoder *)aDecoder {
    if (self = [super init]) {
        _deviceID   = [aDecoder decodeObjectForKey:kYTApiaryDeviceCryptoDeviceIdKey];
        _deviceKey  = [aDecoder decodeObjectForKey:kYTApiaryDeviceCryptoDeviceKeyKey];
        _projectKey = [aDecoder decodeObjectForKey:kYTDeviceCryptoProjectKeyKey];
        _hmacKey    = [aDecoder decodeObjectForKey:kYTDeviceCryptoHMACKeyKey];
        /* Original uses decodeIntForKey, which is not optimal here */
        _hmacLength = [aDecoder decodeIntegerForKey:kYTDeviceCryptoHMACLengthKey];
    }
    return self;
}

- (void)encodeWithCoder:(NSCoder *)aCoder {
    [aCoder encodeObject:_deviceID    forKey:kYTApiaryDeviceCryptoDeviceIdKey];
    [aCoder encodeObject:_deviceKey   forKey:kYTApiaryDeviceCryptoDeviceKeyKey];
    [aCoder encodeObject:_projectKey  forKey:kYTDeviceCryptoProjectKeyKey];
    [aCoder encodeObject:_hmacKey     forKey:kYTDeviceCryptoHMACKeyKey];
    [aCoder encodeInteger:_hmacLength forKey:kYTDeviceCryptoHMACLengthKey];
}

+ (BOOL)supportsSecureCoding {
    return YES;
}

@end

// MARK: -
@implementation NSError (LMNetCryptoError)

+ (instancetype)netCryptoErrorWithMessage:(NSString *)message {
    return [NSError errorWithDomain:@"com.google.ios.youtube.Net.ErrorDomain" code:0 userInfo:@{
        @"message" : message
    }];
}

@end

Hey, great content
am trying now to implement same thing but for android but still couldnt get my head around it, do you have any idea how can i write same code with java or kotlin

@aminerol Yes, this class should be able to be translated to Java with little issue. Do you have any specific questions?

what kind of issues that i could face while translating it to java, knowing that am not that good with objective-c, i tried to decompile the apk for youtube to get a sneak peak but the code is obfuscated and hard to read so still got no luck also can you provide a demo app or code for using that class

The main problem I imagine you'll run into is finding an equivalent for NSMutableURLRequest in Java. I'm not very familiar with Java, but I believe java.net.HttpURLConnection is the closest, and it doesn't provide a way to read the HTTP body, that I know of.
LeptosMusic was supposed to be the example project for using this class, but the method I'm using for getting the protobuf classes isn't great, and isn't public.

Below is my Java translation. It was written to be as close to possible to the code above.

/*
 * This code is a Java translation of LMApiaryDeviceCrypto, originally written in Objective-C.
 *   https://gist.github.com/leptos-null/8792b9c50fddc00cf525ed5055a872dc
 * The LMApiaryDeviceCrypto class was reverse engineered by Leptos from YouTube by Google.
 * 
 * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
 * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
 * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
 * 
 */

package youtube;

import java.util.Arrays;
import java.util.Base64;

import java.nio.ByteBuffer;
import java.nio.charset.StandardCharsets;
import java.net.HttpURLConnection;

import javax.crypto.Cipher;
import javax.crypto.Mac;
import javax.crypto.NoSuchPaddingException;
import javax.crypto.spec.SecretKeySpec;

import javax.security.auth.DestroyFailedException;

import java.security.MessageDigest;
import java.security.InvalidKeyException;
import java.security.NoSuchAlgorithmException;
import java.security.SecureRandom;

public class ApiaryDeviceCrypto {

    private String deviceID;
    private byte[] deviceKey;
    private byte[] projectKey;
    private byte[] hmacKey;
    private int hmacLength;

    public ApiaryDeviceCrypto(byte[] projKey, int signLength) {
        this.hmacLength = signLength;

        final int internalHmacLength = 0x10;
        int projectKeyLength = projKey.length;
        if (projectKeyLength >= internalHmacLength) {
            projectKey = Arrays.copyOfRange(projKey, 0, internalHmacLength);
            this.hmacKey = Arrays.copyOfRange(projKey, internalHmacLength, projectKeyLength - internalHmacLength);
        }
    }

    public boolean setDeviceComponents(String id, String key) throws InvalidKeyException, NoSuchAlgorithmException, NoSuchPaddingException {
        this.deviceKey = this.decryptEncodedString(key);
        this.deviceID = id;
        return true;
    }

    public boolean signConnection(HttpURLConnection connection) throws InvalidKeyException, NoSuchAlgorithmException, DestroyFailedException {
        byte[] urlData = connection.getURL().toString().getBytes(StandardCharsets.UTF_8);
        String signedURL = this.signData(urlData, true, 4);
        byte[] httpBody = null; // TODO: HTTP body of connection
        String signedContent = this.signData(httpBody, false, 20); // CC_SHA1_DIGEST_LENGTH

        String compoundValue = "device_id=" + this.deviceID + ",data=" + signedURL + "content=" + signedContent;
        connection.setRequestProperty("X-Goog-Device-Auth", compoundValue);
        return true;
    }

    private String signData(byte[] data, boolean shouldPad, int hmacLen) throws NoSuchAlgorithmException, InvalidKeyException, DestroyFailedException {
        MessageDigest digest = MessageDigest.getInstance("SHA-1");
        digest.update(this.deviceKey);
        byte[] hashedData = Arrays.copyOf(digest.digest(), 4);

        if (shouldPad) {
            byte[] padData = new byte[data.length + 1];
            System.arraycopy(data, 0, padData, 0, data.length);
            data = padData;
        }

        Mac authCode = Mac.getInstance("HmacSHA1");
        SecretKeySpec signingKey = new SecretKeySpec(this.deviceKey, authCode.getAlgorithm());
        authCode.init(signingKey);

        int appendLength = Math.min(hmacLen, authCode.getMacLength());
        byte zeroByte = 0;

        ByteBuffer newData = ByteBuffer.allocate(1 + hashedData.length + appendLength);
        newData.put(zeroByte);
        newData.put(hashedData);
        newData.put(authCode.doFinal(data), 0, appendLength);
        signingKey.destroy();

        Base64.Encoder stringEncoder = Base64.getEncoder().withoutPadding();
        ByteBuffer encodedData = stringEncoder.encode(newData);
        byte[] encodedBytes;
        if (encodedData.hasArray()) {
            encodedBytes = encodedData.array();
        } else {
            encodedBytes = new byte[encodedData.position()];
            encodedData.get(encodedBytes);
        }
        return new String(encodedBytes, StandardCharsets.UTF_8);
    }

    private byte[] performCrypto(byte[] data, int outputLen, byte[] iv, int operation) throws NoSuchAlgorithmException, NoSuchPaddingException, InvalidKeyException {
        Cipher cryptor = Cipher.getInstance("AES/CTR/NoPadding");
        SecretKeySpec key = new SecretKeySpec(Arrays.copyOf(this.projectKey, 0x10), cryptor.getAlgorithm()); // not sure if this is the correct algorithm to pass
        cryptor.init(operation, key);
        return Arrays.copyOf(cryptor.update(data), outputLen); 
    }

    private byte[] paddedData(byte[] data) {
        final int padMod = 0x10;
        int dataLength = data.length;
        int lengthMod = dataLength % padMod;
        if (lengthMod != 0) {
            byte[] padData = new byte[dataLength + padMod - lengthMod];
            System.arraycopy(data, 0, padData, 0, dataLength);
            return padData;
        } else {
            return data;
        }
    }

    private byte[] projectKeySignature() throws NoSuchAlgorithmException {
        ByteBuffer data = ByteBuffer.allocate(this.hmacKey.length + 0x20);
        final long magic = 0x1000000000000000l;
        data.putLong(magic);
        data.put(this.projectKey);
        data.putLong(magic);
        data.put(this.hmacKey);

        MessageDigest digest = MessageDigest.getInstance("SHA-1");
        digest.update(data);
        return Arrays.copyOf(digest.digest(), 4);
    }

    public byte[] decryptEncodedString(String encoded) throws InvalidKeyException, NoSuchAlgorithmException, NoSuchPaddingException {
        Base64.Decoder stringDecoder = Base64.getDecoder();
        byte[] decoded = stringDecoder.decode(encoded);
        byte firstByte = decoded[0];
        if (firstByte == 0) {
            if (decoded.length > 0xc) {
                byte[] lowPad = this.paddedData(Arrays.copyOfRange(decoded, 5, 8));
                int someVal = decoded.length - this.hmacLength - 0xd;
                if (someVal >= 0) {
                    if (this.verifySignedData(decoded)) {
                        byte[] highPad = this.paddedData(Arrays.copyOfRange(decoded, 0xd, someVal));
                        return this.performCrypto(highPad, someVal, lowPad, Cipher.DECRYPT_MODE);
                    } else {
                        throw new NetCryptoError("Could not verify encrypted data");
                    }
                } else {
                    throw new NetCryptoError("Could not determine cipher");
                }
            } else {
                throw new NetCryptoError("Could not determine initializion vector");
            }
        } else {
            throw new NetCryptoError("Could not determine key sign");
        }
    }

    public String encryptAndEncode(byte[] data) throws InvalidKeyException, NoSuchAlgorithmException, NoSuchPaddingException {
        final byte zeroByte = 0;
        byte[] projectSig = this.projectKeySignature();

        SecureRandom randomGen = new SecureRandom();
        byte[] ivData = new byte[8];
        randomGen.nextBytes(ivData);

        byte[] crypto = this.performCrypto(this.paddedData(data), data.length, this.paddedData(ivData), Cipher.ENCRYPT_MODE);

        int retPre = 1 + projectSig.length + ivData.length + crypto.length;
        ByteBuffer mutData = ByteBuffer.allocate(retPre + this.hmacLength);
        mutData.put(zeroByte);
        mutData.put(projectSig);
        mutData.put(ivData);
        mutData.put(crypto);

        ByteBuffer moreData = ByteBuffer.allocate(retPre + 9);
        byte magicByte = 83;
        moreData.put(magicByte);
        moreData.position(9);
        moreData.put(mutData.array(), 0, retPre);

        Mac hmac = Mac.getInstance("HmacSHA1");
        SecretKeySpec signingKey = new SecretKeySpec(this.hmacKey, hmac.getAlgorithm());
        hmac.init(signingKey);
        assert moreData.hasArray();
        mutData.put(hmac.doFinal(moreData.array()), 0, this.hmacLength);

        Base64.Encoder stringEncoder = Base64.getEncoder().withoutPadding();
        ByteBuffer encodedData = stringEncoder.encode(mutData);
        byte[] encodedBytes;
        if (encodedData.hasArray()) {
            encodedBytes = encodedData.array();
        } else {
            encodedBytes = new byte[encodedData.position()];
            encodedData.get(encodedBytes);
        }
        return new String(encodedBytes, StandardCharsets.UTF_8);        
    }

    private boolean verifySignedData(byte[] data) throws NoSuchAlgorithmException, InvalidKeyException {
        byte[] projectHash = Arrays.copyOfRange(data, 1, 4);
        if (projectHash.equals(this.projectKeySignature())) {
            int lengthDiff = data.length - hmacLength;
            if (lengthDiff >= 0) {
                byte[] highData = Arrays.copyOfRange(data, lengthDiff, hmacLength);
                byte[] lowData = Arrays.copyOfRange(data, 0, lengthDiff);
                ByteBuffer mutData = ByteBuffer.allocate(lengthDiff + 9);

                byte magicByte = 83;
                mutData.put(magicByte);
                mutData.position(9);
                mutData.put(lowData);

                Mac hmac = Mac.getInstance("HmacSHA1");
                SecretKeySpec signingKey = new SecretKeySpec(this.hmacKey, hmac.getAlgorithm());
                hmac.init(signingKey);
                assert mutData.hasArray();
                byte[] checkData = hmac.doFinal(mutData.array());
                return highData.equals(checkData);
            }
        }
        return false;
    }

    // coding/serialization not implemented 

    public class NetCryptoError extends Error {

        // randomly generated
        private static final long serialVersionUID = 5267767227306374604L;

        public NetCryptoError(String message) {
            super(message);
        }

    }
}

@leptos-null thanks for the translation that u made that will give me a headstart to port your implementation of youtube music app from iOS to android, also find a way where you can call the innertube api without going through all of this just change the content-type to application/json and provide the body as json
"context":{ "client":{ "clientName":"ANDROID", "clientVersion":"14.33.56" } }, "browseId":"FEwhat_to_watch",
and you will get response as json.
also for the protobuf u can check this repo where u can find all headers for the youtube music that might help https://github.com/Nosskirneh/ios-app-headers/tree/master/com.google.ios.youtubemusic

Is it possible to do this with php ?

@aminerol thanks for the JSON tip! I’ll probably use that in the project.
I wrote my own tool that output the headers, and implementations, but I wanted to be able to dump the original .protos (see ProtoDump)

@SuhatAkbulak it should be possible to write it in any language. You need some cryptography functions (SHA1 hash, and AES CTR no pad encryption/description), and easy way to manipulate bytes would be helpful.

@leptos-null I'm currently trying to convert the Java version above into Python. I've been using pyaes, hmac and hashlib.sha1 for the crypto, and python-bytebuffer (https://github.com/alon-sage/python-bytebuffer) for working with bytes. I can't seem to get it working, and was wondering if you'd be able to help with a Python translation?

@tombulled I can try to help with any specific issues you have.

@leptos-null Thank you for your speedy response, I've created a gist so as not to clutter up yours (https://gist.github.com/tombulled/d313c54a0681fcf0ba6d8092f11411e6). I've had to redact some values to ensure I'm not leaking any information. Please do take a look at it and I'd really appreciate any help you're able to give!

I reversed the last version in android, and it is quite the same. I made it in PHP, if some needs it, i implemented also fetch player request. Feel free to clone my repo