Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: Implement mutating versions of @set and @lens (#71) #92

Closed
wants to merge 5 commits into from
Closed

WIP: Implement mutating versions of @set and @lens (#71) #92

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
e1387c3
use ConstructionBase
jw3126 Oct 1, 2019
c27c4da
bump version to 0.5.0
jw3126 Oct 2, 2019
d82aef5
Merge branch 'constbase' into bangbang
Oct 8, 2019
a0bd698
add preliminary implementation of #71
Oct 8, 2019
5035991
switch to MutationPolicy
Oct 9, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion Project.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,8 +1,10 @@
name = "Setfield"
uuid = "efcf1570-3423-57d1-acb7-fd33fddbac46"
version = "0.4.1"
version = "0.5.0"

[deps]
BangBang = "198e06fe-97b7-11e9-32a5-e1d131e6ad66"
ConstructionBase = "187b0558-2788-49d3-abe0-74a17ed4e7c9"
MacroTools = "1914dd2f-81c6-5fcd-8719-6d5c9610ff09"

[compat]
Expand Down
3 changes: 2 additions & 1 deletion src/Setfield.jl
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
__precompile__(true)
module Setfield
using MacroTools
using MacroTools: isstructdef, splitstructdef, postwalk
using MacroTools: isstructdef, splitstructdef
using BangBang: setproperty!!, setindex!!
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BangBang is not a super small library and it probably will accumulate more code over time, as it has to re-implement some Base/stdlib functions. I'm not sure if it is a good approach for Setfield.jl to depend on it. Maybe we should factor out setproperty!! and setindex!! to ConstructionBase.jl or some small library under JuliaObjects? (Also, we probably should create a new API setproperties!! instead.)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense! What's your timeline on registering ConstructionBase.jl?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are just waiting for it JuliaRegistries/General#4063

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, awesome!! Glad to hear.

Copy link
Author

@colinxs colinxs Oct 8, 2019 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we do refactor, need that block this PR? I'm happy to make the changes, just looking to get this merged sooner-than-later so I can start developing around it :).

Also, is #91 just waiting on JuliaRegistries/General#4063?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just pushed the changes to use MutationPolicy.

In principal, I like the idea of making Setfield extensible. At one point I was considering manipulating deeply nested C structs using unsafe_store! for cases where (1) the compiler can't optimize away object construction and (2) thread-safe field manipulation. I recently found mchristianl/MemoryMutate.jl which does just this and could be re-implemented in terms of Setfield.jl.

The separation of BangBang is obviously up to you all.

My selfish opinion would be if you want to refactor BangBang, then make Setfield extensible, otherwise either is fine :)

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • I am fine with Setfield depending on BangBang temporarily if we have a plan how to fix that in near future.
  • If we move the required things from BangBang to ConstructionBase, how much code/dependencies would that add to ConstructionBase @tkf?
  • I like the idea to split @set more cleanly in a "parsing" step and an "interpretation" step that can be overloaded by other macros. But I feel developing this is quite some effort and should be done independently of this PR.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • I am fine with Setfield depending on BangBang temporarily if we have a plan how to fix that in near future.

I think fixing it (= separating out the base part of BangBang) is hard.

  • If we move the required things from BangBang to ConstructionBase, how much code/dependencies would that add to ConstructionBase

Reviewing my code in BangBang.jl, I realized that there are quite a few interface methods in BangBang.jl (may, possible, pure, ismutable, ismutablestruct, and _asbb). I think it would be too much to add those methods as-is in ConstructionBase.jl. (Although I think it makes sense to have ismutablestruct(::Type{<:StructType}) and even ismutable(::Type{<:CollectionType}) there.) What ConstructionBase.jl has at the moment are a few functions that are definitely the basic things you want. However, I feel what BangBang.jl has are rather opinionated set of functions (even though I think this is a nice direction to explore).

  • But I feel developing this is quite some effort and should be done independently of this PR.

I do agree this is a hard task. But I think it's the most reasonable option for implementing @set!! and @set! (If I were forced choose refactoring BangBang.jl or making Setfield.jl extensible, I'll definitely choose the latter).

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did a POC here.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I responed JuliaObjects/ConstructionBase.jl#25 (comment)


include("lens.jl")
include("sugar.jl")
Expand Down
4 changes: 2 additions & 2 deletions src/experimental.jl
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
module Experimental
using Setfield
using Setfield: constructor_of
using ConstructionBase: constructorof
import Setfield: get, set
export MultiPropertyLens

Expand Down Expand Up @@ -32,7 +32,7 @@ end
end
Expr(:block,
Expr(:meta, :inline),
Expr(:call, :(constructor_of($T)), args...)
Expr(:call, :(constructorof($T)), args...)
)
end

Expand Down
109 changes: 37 additions & 72 deletions src/lens.jl
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,10 @@
export Lens, set, get, modify
export @lens
export set, get, modify
using ConstructionBase
export setproperties
export constructorof


import Base: get
using Base: setindex, getproperty
Expand Down Expand Up @@ -81,6 +84,11 @@ Replace a deeply nested part of `obj` by `val`. See also [`Lens`](@ref).
"""
function set end

abstract type BangStyle end
struct NoBang <: BangStyle end
struct OneBang <: BangStyle end
struct TwoBang <: BangStyle end

@inline function modify(f, obj, l::Lens)
old_val = get(obj, l)
new_val = f(old_val)
Expand All @@ -92,8 +100,10 @@ get(obj, ::IdentityLens) = obj
set(obj, ::IdentityLens, val) = val

struct PropertyLens{fieldname} <: Lens end
struct PropertyLens!{fieldname} <: Lens end
struct PropertyLens!!{fieldname} <: Lens end
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would implement it using something like this:

abstract type MutationPolicy end
struct Immutable <: MutationPolicy end  # no bang
struct AlwaysMutate <: MutationPolicy end  # !
struct TryMutate <: MutationPolicy end  # !!

struct PropertyLens{fieldname, TP <: MutationPolicy} <: Lens
    policy::TP
end

PropertyLens(fieldname, policy = Immutable()) =
    PropertyLens{fieldname, typeof(policy)}(policy)

It would also be handy define accessor functions

MutationPolicy(::Lens) = Immutable()  # should it be this?
MutationPolicy(l::PropertyLens) = l.policy

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah that seems more concise (and also replaces the fairly redundant BangStyle trait). To maintain PropertyLens being a Singleton, what about:

struct PropertyLens{fieldname, TP} end
PropertyLens(fieldname, policy::MutationPolicy = Immutable()) = PropertyLens{fieldname, policy}()
MutationPolicy(::PropertyLens{<:Any, TP}) = TP

Also, just curious, what does TP stand for?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To maintain PropertyLens being a Singleton

It would be a singleton if plicy is a singleton:

struct Demo1 end
struct Demo2{T}
    demo::T
end

julia> sizeof(Demo2(Demo1()))

julia> Base.issingletontype(typeof(Demo2(Demo1())))
true

If we want to enforce the policy to be a singleton, I think your approach would be better.

what does TP stand for?

I was thinking Type-of-Policy. It can have a better name, too :)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ahh, I didn't know that! I was operating off of the definition in the docs:

Immutable composite types with no fields are singletons

which I took to imply "singleton if-and-only-if immutable composite type with no fields". Thanks for the explanation!

Perhaps MP for Mutation Policy?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe? We can also call it simply Policy, as done in e.g. Broadcasted: https://github.com/JuliaLang/julia/blob/d5d5718b85d829aff6c61ab23ad2d91a50f8688b/base/broadcast.jl#L169-L173

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the verbosity.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • I would rename Immutable -> NeverMutate. It is more consistent with the other variants and we might want the name Immutable for another trait.
  • I think MutationPolixy(l::Lens) = NeverMutate() is the right default. I think Setfield still is mainly about manipulation of immutable structs.
  • The design should be so that an ad hoc defined lens only needs to implement get(obj, lens) and set(obj, lens, val) and still works if it is not interested in mutating things.


function get(obj, l::PropertyLens{field}) where {field}
function get(obj, l::Union{PropertyLens{field}, PropertyLens!{field}, PropertyLens!!{field}}) where {field}
getproperty(obj, field)
end

Expand All @@ -103,66 +113,9 @@ end
:(setproperties(obj, ($field=val,)))
)
end
@inline set(obj, l::PropertyLens!{field}, val) where {field} = setproperty!(obj, field, val)
@inline set(obj, l::PropertyLens!!{field}, val) where {field} = setproperty!!(obj, field, val)

@generated constructor_of(::Type{T}) where T =
getfield(parentmodule(T), nameof(T))

function assert_hasfields(T, fnames)
for fname in fnames
if !(fname in fieldnames(T))
msg = "$T has no field $fname"
throw(ArgumentError(msg))
end
end
end

"""
setproperties(obj, patch)

Return a copy of `obj` with attributes updates accoring to `patch`.

# Examples
```jldoctest
julia> using Setfield

julia> struct S;a;b;c; end

julia> s = S(1,2,3)
S(1, 2, 3)

julia> setproperties(s, (a=10,c=4))
S(10, 2, 4)

julia> setproperties((a=1,c=2,b=3), (a=10,c=4))
(a = 10, c = 4, b = 3)
```
"""
function setproperties end

@generated function setproperties(obj, patch)
assert_hasfields(obj, fieldnames(patch))
args = map(fieldnames(obj)) do fn
if fn in fieldnames(patch)
:(patch.$fn)
else
:(obj.$fn)
end
end
Expr(:block,
Expr(:meta, :inline),
Expr(:call,:(constructor_of($obj)), args...)
)
end

@generated function setproperties(obj::NamedTuple, patch)
# this function is only generated to force the following check
# at compile time
assert_hasfields(obj, fieldnames(patch))
Expr(:block,
Expr(:meta, :inline),
:(merge(obj, patch))
)
end

struct ComposedLens{LO, LI} <: Lens
outer::LO
Expand Down Expand Up @@ -219,13 +172,26 @@ end
struct IndexLens{I <: Tuple} <: Lens
indices::I
end
struct IndexLens!{I <: Tuple} <: Lens
indices::I
end
struct IndexLens!!{I <: Tuple} <: Lens
indices::I
end

Base.@propagate_inbounds function get(obj, l::IndexLens)
Base.@propagate_inbounds function get(obj, l::Union{IndexLens, IndexLens!, IndexLens!!})
getindex(obj, l.indices...)
end

Base.@propagate_inbounds function set(obj, l::IndexLens, val)
setindex(obj, val, l.indices...)
end
Base.@propagate_inbounds function set(obj, l::IndexLens!, val)
setindex!(obj, val, l.indices...)
end
Base.@propagate_inbounds function set(obj, l::IndexLens!!, val)
setindex!!(obj, val, l.indices...)
end

"""
ConstIndexLens{I}
Expand All @@ -251,14 +217,21 @@ true
```
"""
struct ConstIndexLens{I} <: Lens end
struct ConstIndexLens!{I} <: Lens end
struct ConstIndexLens!!{I} <: Lens end

Base.@propagate_inbounds get(obj, ::ConstIndexLens{I}) where I = obj[I...]
Base.@propagate_inbounds get(obj, ::Union{ConstIndexLens{I}, ConstIndexLens!{I}, ConstIndexLens!!{I}}) where I = obj[I...]

Base.@propagate_inbounds set(obj, ::ConstIndexLens{I}, val) where I =
setindex(obj, val, I...)
Base.@propagate_inbounds set(obj, ::ConstIndexLens!{I}, val) where I =
setindex!(obj, val, I...)
Base.@propagate_inbounds set(obj, ::ConstIndexLens!!{I}, val) where I =
setindex!!(obj, val, I...)

# TODO: Do we want/need !/!! for ConstIndexLens? If so, should below also include !! lens?
@generated function set(obj::Union{Tuple, NamedTuple},
::ConstIndexLens{I},
::Union{ConstIndexLens{I}, ConstIndexLens!!{I}},
val) where I
if length(I) == 1
n, = I
Expand All @@ -278,15 +251,6 @@ Base.@propagate_inbounds set(obj, ::ConstIndexLens{I}, val) where I =
end
end

struct DynamicIndexLens{F} <: Lens
f::F
end

Base.@propagate_inbounds get(obj, I::DynamicIndexLens) = obj[I.f(obj)...]

Base.@propagate_inbounds set(obj, I::DynamicIndexLens, val) =
setindex(obj, val, I.f(obj)...)

"""
FunctionLens(f)
@lens f(_)
Expand Down Expand Up @@ -333,6 +297,7 @@ FunctionLens(f) = FunctionLens{f}()

get(obj, ::FunctionLens{f}) where f = f(obj)

Base.@deprecate constructor_of(T) constructorof(T)
Base.@deprecate get(lens::Lens, obj) get(obj, lens)
Base.@deprecate set(lens::Lens, obj, val) set(obj, lens, val)
Base.@deprecate modify(f, lens::Lens, obj) modify(f, obj, lens)
86 changes: 42 additions & 44 deletions src/sugar.jl
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
export @set, @lens, @set!
export @set, @set!, @set!!, @lens, @lens!, @lens!!
using MacroTools

"""
Expand Down Expand Up @@ -29,7 +29,7 @@ T(T(2, 3), 2)
```
"""
macro set(ex)
atset_impl(ex, overwrite=false)
atset_impl(ex, NoBang())
end

"""
Expand All @@ -47,38 +47,18 @@ julia> t
(a = 2,)
"""
macro set!(ex)
atset_impl(ex, overwrite=true)
atset_impl(ex, OneBang())
end

is_interpolation(x) = x isa Expr && x.head == :$

foldtree(op, init, x) = op(init, x)
foldtree(op, init, ex::Expr) =
op(foldl((acc, x) -> foldtree(op, acc, x), ex.args; init=init), ex)

need_dynamic_lens(ex) =
foldtree(false, ex) do yes, x
yes || x === :end || x === :_
end

replace_underscore(ex, to) = postwalk(x -> x === :_ ? to : x, ex)

function lower_index(collection::Symbol, index, dim)
if isexpr(index, :call)
return Expr(:call, lower_index.(collection, index.args, dim)...)
elseif index === :end
if dim === nothing
return :($(Base.lastindex)($collection))
else
return :($(Base.lastindex)($collection, $dim))
end
end
return index
macro set!!(ex)
atset_impl(ex, TwoBang())
end

function parse_obj_lenses(ex)
is_interpolation(x) = x isa Expr && x.head == :$

function parse_obj_lenses(ex, bang::BangStyle)
if @capture(ex, front_[indices__])
obj, frontlens = parse_obj_lenses(front)
obj, frontlens = parse_obj_lenses(front, bang)
if any(is_interpolation, indices)
if !all(is_interpolation, indices)
throw(ArgumentError(string(
Expand All @@ -87,21 +67,21 @@ function parse_obj_lenses(ex)
end
index = esc(Expr(:tuple, [x.args[1] for x in indices]...))
lens = :(ConstIndexLens{$index}())
elseif any(need_dynamic_lens, indices)
@gensym collection
indices = replace_underscore.(indices, collection)
dims = length(indices) == 1 ? nothing : 1:length(indices)
lindices = esc.(lower_index.(collection, indices, dims))
lens = :(DynamicIndexLens($(esc(collection)) -> ($(lindices...),)))
else
index = esc(Expr(:tuple, indices...))
lens = :(IndexLens($index))
end
elseif @capture(ex, front_.property_)
obj, frontlens = parse_obj_lenses(front)
lens = :(PropertyLens{$(QuoteNode(property))}())
obj, frontlens = parse_obj_lenses(front, bang)
if bang isa NoBang
lens = :(PropertyLens{$(QuoteNode(property))}())
elseif bang isa OneBang
lens = :(PropertyLens!{$(QuoteNode(property))}())
elseif bang isa TwoBang
lens = :(PropertyLens!!{$(QuoteNode(property))}())
end
elseif @capture(ex, f_(front_))
obj, frontlens = parse_obj_lenses(front)
obj, frontlens = parse_obj_lenses(front, bang)
lens = :(FunctionLens($(esc(f))))
else
obj = esc(ex)
Expand All @@ -110,8 +90,8 @@ function parse_obj_lenses(ex)
obj, tuple(frontlens..., lens)
end

function parse_obj_lens(ex)
obj, lenses = parse_obj_lenses(ex)
function parse_obj_lens(ex, bang::BangStyle)
obj, lenses = parse_obj_lenses(ex, bang)
lens = Expr(:call, :compose, lenses...)
obj, lens
end
Expand All @@ -133,12 +113,12 @@ struct _UpdateOp{OP,V}
end
(u::_UpdateOp)(x) = u.op(x, u.val)

function atset_impl(ex::Expr; overwrite::Bool)
function atset_impl(ex::Expr, bang::BangStyle)
@assert ex.head isa Symbol
@assert length(ex.args) == 2
ref, val = ex.args
obj, lens = parse_obj_lens(ref)
dst = overwrite ? obj : gensym("_")
obj, lens = parse_obj_lens(ref, bang)
dst = gensym("_")
val = esc(val)
ret = if ex.head == :(=)
quote
Expand Down Expand Up @@ -188,7 +168,25 @@ julia> set(t, (@lens _[1]), "1")

"""
macro lens(ex)
obj, lens = parse_obj_lens(ex)
obj, lens = parse_obj_lens(ex, NoBang())
if obj != esc(:_)
msg = """Cannot parse lens $ex. Lens expressions must start with @lens _"""
throw(ArgumentError(msg))
end
lens
end

macro lens!(ex)
obj, lens = parse_obj_lens(ex, OneBang())
if obj != esc(:_)
msg = """Cannot parse lens $ex. Lens expressions must start with @lens _"""
throw(ArgumentError(msg))
end
lens
end

macro lens!!(ex)
obj, lens = parse_obj_lens(ex, TwoBang())
if obj != esc(:_)
msg = """Cannot parse lens $ex. Lens expressions must start with @lens _"""
throw(ArgumentError(msg))
Expand Down
Loading