We cache on the AST node side as this is easier to track a position, we just have to take care to wrap the values in a handle to make sure they are not garbage collected.