Revised Answer The correct SQL came to me last night. Here's my revised SQL. I left my original answer for completeness and disclosure (though you could see it from the history of the gist).
SELECT
p.PersonId, p.NameFirst, p.NameLast,
(select top 1 OrderId from Order o where o.PersonId=p.PersonId Order by OrderDateTime DESC) as LastOrderId,
(select max(OrderDateTime) from Order o where o.PersonId=p.PersonId) as LastOrderDateTime
FROM Person p
ORDER BY p.PersonId DESC