Dan Zheng dan-zheng

Python interoperability

Check out this tutorial demonstrating Python interoperability in Swift.

Python library version details

`PythonLibrary.useVersion`

This function is intended to set the Python version before any references to the global PythonInterface called Python - it cannot be used to switch Python versions dynamically.

I tried to define a generic Adam optimizer using ElementaryFunctions, Differentiable, and VectorProtocol (latter two are being incubated on tensorflow branch).

It doesn't work due to missing scalar-vector and vector-vector operations.

$ swift adam.swift
adam.swift:82:95: error: binary operator '+' cannot be applied to operands of type 'Model.TangentVector' and 'Model.TangentVector.VectorSpaceScalar'
        model.move(along: -stepSize * firstMoments / (Model.TangentVector.sqrt(secondMoments) + epsilon))
                                                      ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ^ ~~~~~~~
adam.swift:82:95: note: expected an argument list of type '(Self, Self)'

Retroactive derivative registration: register derivatives for functions in other modules.

Previously:

If module A defines func foo, then its derivatives must be in the same module.

With retroactive derivative registration:

If module A defines func foo;
And if module B imports module A and defines @differentiating(foo) func derivativeFoo:
Then module C can import modules A and B and differentiate func foo (e.g. via differentiation APIs).

tensorflow/swift-apis compilation is painfully slow. It's probably due to type-checking.

Swift compilation debugging tips here. swift build -Xswiftc -Xfrontend -Xswiftc -debug-time-function-bodies results below.

# Worst offenders, time in milliseconds.
(1767, "global function 'gelu'", '/Users/danielzheng/swift-apis/Sources/TensorFlow/Operators/Math.swift:1225:13')
(1767, "global function 'gelu'", '/Users/danielzheng/swift-apis/Sources/TensorFlow/Operators/Math.swift:1225:13')
(1767, "global function 'gelu'", '/Users/danielzheng/swift-apis/Sources/TensorFlow/Operators/Math.swift:1225:13')

Keras-style Sequential in Swift for TensorFlow.

The current Sequential in Swift for TensorFlow does not support shape inference. Previous layers' output sizes are redundantly repeated as next layers' input sizes. This increases user burden and adds room for error.

let modelOld = Sequential {

	import torch
	import tangent
	from tangent import grad

	def nested_loop(x):
	outer = x
	for _ in range(1, 3):
	outer = outer * x
	inner = outer