Skip to content

Instantly share code, notes, and snippets.

@AlexNisnevich
Created September 25, 2013 06:08
Show Gist options
  • Save AlexNisnevich/6695710 to your computer and use it in GitHub Desktop.
Save AlexNisnevich/6695710 to your computer and use it in GitHub Desktop.
[num_features, num_states] = size(backup_states_phi); % [22, 92]
num_actions = size(R{1}{1}); % 40
num_blocks = block_idx; % 7
mu0 = ones(num_states, 1) / num_states;
% cvx_solver sedumi
cvx_begin
variable theta(num_features)
variable V(num_states, num_blocks)
minimize sum(dot(mu0, (transpose(theta) * backup_states_phi))) + C * norm(theta)
subject to
for b=1:num_states,
transpose(theta) * backup_states_phi(:,b) >= mean(V(b,:));
for a=1:num_actions,
for k=1:num_blocks,
V(b,k) >= R{b}{k}(a) + discount * transpose(theta) * phis_next{b}{k}(:,a);
end
end
end
cvx_end
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment