Support for Xaml/Data-binding and Dynamic Content

[Mike-E-angelo]

I am opening this ticket in response to this thread to provide better communication around this issue. The ask is to provide better designer/tooling support around Tile Notifications and to explore better ways of doing so with existing environments.

My first question is: how are Tile Notifications/Toasts ultimately rendered in UWP/WinRT? Is it using some sort of GDI mechanism? Or is it actually rendered as UWP UI elements? Or COM elements? Knowing this will help me figure out a good designer strategy around it.

I would also like to have a little more context around the XML Document that is passed into a TileNotification constructor. This does seem like an anti-pattern (or at the very least, poor guidance for other developers to follow), so I am wanting to understand how this document is used after it is passed behind the Curtain of COM. Unfortunately, with UWP, after it passes the Curtain of COM, it is impossible to see what happens to something. In .NET, we are able to trace things to get a better understanding of the system and overall design, but in UWP we lose this power/quality.

Thank you for any perspective/information!

[andrewleader]

Rendering adaptive tile XML works like this: XML comes in, gets converted to Windows.UI.Xaml elements, then a PNG bitmap is generated from the elements, and then the PNG is displayed on the tile.

I think toasts are actually rendered live in the shell (so Splash for Mobile and XAML for desktop).

For converting XML to the Windows XAML elements, I've already written something called the Tile Template Visualizer. It replicates 99% of how Start renders the XML. It takes a lot of time (way more time than writing the NotificationsExtensions library) to replicate all the features of adaptive XML (have to implement dropping off of groups, etc).

However, what we really want is to get permission to use the actual DLL that Windows uses to render the XML, so that it could be 100% accurate. And then maybe the Visual Studio designer could also use that?

Either way, for converting the XML to visuals, I've got an existing large code base that we could leverage. You should message me privately and we can go from there.

And as for what happens to the XML Doc - we store the exact XML, and then parse it when it's needed.

[Mike-E-angelo]

Thanks for letting me know. I found out about the following vote today: XAML for live tiles -- so this is on the radar already in several forms.

Based on what the team has to say in this vote, it is clear that I have some research to do, so please forgive me for my ignorance as I learn about everything here.

I guess what I am primarily/most concerned with is the use of XmlDocument to begin with. I am a bit hung up on this, so you will have to bear with me before getting to the good stuff. :) If the XmlDocument gets parsed into Xaml, then why not pass a UWP object into TileNotification? Or better yet, why not allow users to define the TileNotification explicitly and then create the XML document from that in COM? Passing in an XmlDocument just really seems like a questioable design decision, as there is no validation performed on the document, and it can be wrong/invalid in so many different ways (leading to obscure, difficult to understand UWP errors that it is unfortunately becoming known for). Additionally, if it is supposed to represent an object, then the object should be used instead as it (obviously) is a well-known contract that has an early-bound schema and strong-typing (read: less bugs and room for error :)). Not to mention, it is a PITA to define (and subsequently read) an XmlDocument and accurately pull information from it.

Like I mentioned, I am wanting to make sure I understand the context around this design first and foremost and am wondering if there is something I am missing. It just does not seem like something a first class API from Microsoft would provide (and I cannot think of any API I have ever seen that uses it).

It would indeed be nice to know how exactly that XML is being rendered and the DLL/classes that are used to render it. Unfortunately, I do not have your PM/email, unless I am overlooking something. Is it your handle on the MSDN boards?

[andrewleader]

Don't worry, we (PM's on the developer platform notification team) agree that passing in an XmlDocument to the local API is silly. For push notifications using XML makes sense (you have to serialize it somehow), but for local notifications, it's pretty ridiculous. The local API should expose an object that constructs the notification (just like NotificationsExtensions does), and that's what the API should accept.

To contact me - I assumed GitHub would have a private message system. Apparently not. Email me at {my_github_username}@microsoft.com

[Mike-E-angelo]

OK I cannot tell you how relieved I am to hear you say this. I know you mentioned in MSDN that XML is old skool, but I didn't know if you really knew what that meant. ;)

At the very least, my feeling is that this constructor should be deprecated (truly, I am horrified by it, LOL!) and the TileNotification should become a design-time component (or at least designer-friendly) instead.

I would be interested in knowing how the push notifications work as well, as the XML used does not look like a serialized object but does look like a DOM. Ideally, all XML used at any time should just be a serialized object and no one anywhere should ever have to load/selectnodes ever again, amen. :)

GitHub did indeed used to have PMs, but closed that feature and moved everything to Twitter and other means. I agree that it would be nice to have. I will write you at your address and we can continue from there. Thanks, Andrew!

[andrewleader]

Oh yeah another concern about making a change like this is not breaking the API surface for existing users. Ideally, people who have already downloaded the NuGet package should be able to upgrade to the new version and their code still works without any changes.

If necessary, we could make some breaking changes, since the NuGet package hasn't been downloaded that much... but I'd prefer to try and avoid that.

[Mike-E-angelo]

Yeah I hear that concern. I definitely intend on breaking a few eggs here. :P But at the very least all the existing tests will pass. Are you open to maybe upping the version on this, perhaps? Maybe NotificationsExtensions 1.5 or 2.0?

[andrewleader]

Yeah I think NuGet probably supports a "breaking change" versioning scheme? So that would be an option that I would be ok with using.

I noticed on Scott's AdaptiveTileExtensions you had to change the nullable objects to just be plain objects yeah? Luckily with mine almost every property directly uses the enums, they're not nullable, so those changes shouldn't be necessary? If there's any drastic changes you're looking at, check the design with me before spending time implementing them.

[Mike-E-angelo]

Luckily with mine almost every property directly uses the enums, they're not nullable, so those changes shouldn't be necessary?

Ugh! What a terrible terrible limitation that is in existing UWP Xaml. It makes me wonder if someone was paid to make it that bad. :p

The concern with any DSL is to accurately capture the conceptual model that end consumers will be interacting with. I do not think that is captured at all in the current version. Not accurately, at least. For instance, there are a lot of "Do not use this field in a WideTile binding". I see that and wonder... why not have a WideTileBinding class? That way you guide the end consumer into the appropriate path. Otherwise, they will hang themselves. :P

Another design I've seen is the LockDetailedStatus* properties (also WideTile-specific, so should be moved to a WideTileBinding class). To me this seems like it should be either a new class to combine these values (like an address) or even an extended StringCollection that throws an exception after 3 items? Or even a combination of the two.

There are also interfaces that do not have any members (which is a violation of FxCop) and multiple places where interfaces are used, but aren't. Example:

if (child is TileText)
    return (child as TileText).ConvertToElement();

else if (child is TileImage)
    return (child as TileImage).ConvertToElement();

else if (child is TileGroup)
    return (child as TileGroup).ConvertToElement();

Should be:

var item = child as IElementAware;
if ( item != null )
{
  return item.ConvertToElement();
}

As for the null's, TileVisual.Version is null, so we will be running into that problem there. Essentially, any property that is null is an risk with UWP Xaml serialization (still cannot get over it). However, I still want to play around with the FrameworkElement path to see if this can be mitigated. Unfortunately, this means that Xaml designer support will only work in UWP projects and not ASP.NET Core projects. There is a discussion out there to bring System.Xaml into a Core .NET assembly, so if you can pull any strings over there to make that happen (or at the very least, create awareness around it), it would be most appreciated! :)

[Mike-E-angelo]

Also, in TileVisual, I would almost consider a TileBindingCollection that accepts items of a type only once. That way you can add as many as you need but only one of a certain type. If a new tile type is created, then you only have to worry about creating it (1 place to manage/maintain) and not having to update wherever it is used/defined (2+ places to manage/maintain).

[andrewleader]

The concern with any DSL is to accurately capture the conceptual model that end consumers will be interacting with. I do not think that is captured at all in the current version. Not accurately, at least. For instance, there are a lot of "Do not use this field in a WideTile binding". I see that and wonder... why not have a WideTileBinding class? That way you guide the end consumer into the appropriate path. Otherwise, they will hang themselves. :P

Yep, I addressed a lot of that with NotificationsExtensions. For example, with the XML, you can technically put a hint-crop on a background image, even though that's not supported. With my library, that's impossible. If you find any issues with NotificationsExtensions where I failed to do that, tell me.

Another design I've seen is the LockDetailedStatus* properties (also WideTile-specific, so should be moved to a WideTileBinding class). To me this seems like it should be either a new class to combine these values (like an address) or even an extended StringCollection that throws an exception after 3 items? Or even a combination of the two.

There's a few drawbacks about creating SmallTileBinding, MediumTileBinding, WideTileBinding classes... what if you want to apply the same binding to both medium, wide, and large tiles? If the TileWide only accepts WideTileBinding, then you can't share bindings between them. Putting the LockDetailedStatus property on the TileVisual element was the most elegant solution I could think of for that. Also, if we did the separate binding classes, there's no good way to implement that in WinRT since you can't extend classes. You would have to make an ISmallTileBinding, IMediumTileBinding, etc interfaces, each with copy/pasted code, and then you could have internal SmallTileBinding, MediumTileBinding, etc classes that use inheritance, but then for the user to initialize these bindings, you would need a factory, which means that users can't use the object-initializer syntax, which is a huge negative... The only other choice is literally creating 4 separate classes for each binding with copied code everywhere, not using inheritance at all. Having to support WinRT (so that JavaScript apps can use it) is a real pain. That's the reason why some of my code doesn't have the best design practices.

Combining the 3 lock properties in a class, or making it a string collection... yeah, I thought about it, but instead chose to remain consistent with the XML itself.

There are also interfaces that do not have any members (which is a violation of FxCop)

Have a better way for a list to only accept objects of a certain type? Like accepting both TileText and TileImage? Those two classes share nothing else in common to the API-facing end-user.

and multiple places where interfaces are used, but aren't. Example:

That's because I have to support WinRT, which makes it impossible to do what you're saying I should do, unless I expose the Element_Tile* classes as public, which I did not want to do. Because in WinRT, a public class cannot implement an internal interface, so the interface needs to be public, which means all methods of the interface need to be public, and since that method would be returning an Element_Tile* object, the Element_Tile* class needs to be public, and then it's improperly exposed.

As for the null's, TileVisual.Version is null, so we will be running into that problem there. Essentially, any property that is null is an risk with UWP Xaml serialization (still cannot get over it). However, I still want to play around with the FrameworkElement path to see if this can be mitigated. Unfortunately, this means that Xaml designer support will only work in UWP projects and not ASP.NET Core projects. There is a discussion out there to bring System.Xaml into a Core .NET assembly, so if you can pull any strings over there to make that happen (or at the very least, create awareness around it), it would be most appreciated! :)

It'd definitely be optimal if the designer worked on all projects. However, would generating via code still work on ASP.NET Core? Even though the objects inherit from FrameworkElement? That would be a deal breaker if they can't even generate notifications at all.

Also, in TileVisual, I would almost consider a TileBindingCollection that accepts items of a type only once. That way you can add as many as you need but only one of a certain type. If a new tile type is created, then you only have to worry about creating it (1 place to manage/maintain) and not having to update wherever it is used/defined (2+ places to manage/maintain).

I disagree. The reason I chose the TileSmall, TileMedium, TileWide, and TileLarge properties is so that it makes it explicit to the developer that "Hey, here's all these sizes, they exist, and you should support them!!". Also, it allows devs to take one TileBinding object and assign it to both medium, wide, and large.

Now, that decision was made with the focus of this code being generated from C#, not in XAML. In a XAML-defined way, it possibly is more elegant to have the TileBindingCollection.

[Mike-E-angelo]

Wow, I totally spaced on this discussion. Real Life caught up with me in a big way. Sorry for the delay here!

Having to support WinRT (so that JavaScript apps can use it) is a real pain. That's the reason why some of my code doesn't have the best design practices.

I am actually unfamiliar with this, and you mention not being able to extend classes... is this a JavaScript WinRT limitation? If so, that is pretty terrible. I mean, WinJS is a pretty terrible idea to begin with, but not being able to extend classes altogether is just plain ridiculous. Who in their right mind would use that API unless MSFT was paying them to do it? (No, please don't actually answer that, LOL!)

but instead chose to remain consistent with the XML itself.

I guess the point here if anything that I am trying to make is... that we shouldn't let XML drive the model. The model should drive the XML. :) If there is anything my b!tching and complaining has accomplished going forward, it is that point exactly.

would generating via code still work on ASP.NET Core?

That is a great question. ASP.NET Core (or what is supposed to be renamed Core .NET) really has its own idea of how projects and designers should behave. Even though we are told the experience is going to be "lit up" in Visual Studio, from the looks of the early betas, it appears to be nothing but that, and in fact features that "just worked" are being broken all over again. You would think/expect that a Core.NET project will allow to edit/view/design a Xaml file just like every other project type ever made in the history of Visual Studio system but as it currently stands, that indeed is not the case (but should be 😄).

But I digress, the FrameworkElement idea involved some design ideas I wanted to try and flesh out, mostly involving interfaces and shared libraries. Although now seeing the dependency of WinJS as the apparent liability that this brings to the table, I am having second thoughts around this.

The reason I chose the TileSmall, TileMedium, TileWide, and TileLarge properties is so that it makes it explicit to the developer that "Hey, here's all these sizes, they exist, and you should support them!!"

Fair enough. I can dig that. Thank you for sharing your perspective and pointing that out.

So, after putting some thought into this, I am beginning to feel that I have bitten off more than I can chew here. I think the changes that I want to bring about are more for the next version/vNext. I started this off with the AdaptiveTileExtensions to show off that all of this can (and should) be done with Xaml and a better designer experience around what is already offered and possible within Visual Studio. The drive around that was personal as when I first saw the XML it was definitely a "WTF" moment and knew that I would create something very similar to the PR I ultimately created for my own personal endeavors.

If anything, I am thinking this issue should be tagged for a vNext as it really appears/seems that all of the seemingly curious design decisions are based around the even more seemingly curious spectacle of WinJS and WinRT. :) If the internal groups do somehow get aligned and you do manage to get centered on a better design paradigm, I would be more than willing to assist with this in that endeavor as well.

Additionally, I would also be willing to further assist here in a capacity that improves everything but WinJS. Seriously... is anyone even using WinJS? ;) Amazing how Silverlight gets the hammer but WinJS somehow manages to survive and continue to seem like a good idea.

[andrewleader]

Hahah no problem :P

you mention not being able to extend classes... is this a JavaScript WinRT limitation?

Yep, it's a JavaScript limitation, extending classes doesn't really exist in JavaScript (apparently they have prototypes, but I guess that can't translate - I'm not much of a JavaScript expert myself).

we shouldn't let XML drive the model. The model should drive the XML. :)

Yeah, true, but if people are used to the XML, sometimes keeping things familiar is the smartest choice. If most documentation describes notifications in the XML syntax, then people would find it frustrating translating the XML to this library's format. I'm working on adding NotificationsExtensions syntax to all the new tile/toast documentation I've published to help with that. But the fact is, the XML came first, and the XML is still the truth, and the XML is the only thing natively supported in the platform, and because of that I feel the library needs to maintain a decent amount of consistency with the XML. If the library was natively included in the Windows API's (like it should be), then things would be different. But there's still going to be tons of devs using raw XML, and still going to be people sharing XML snippets, and I don't want this library to look completely foreign compared to the XML.

I am beginning to feel that I have bitten off more than I can chew here... I started this off with the AdaptiveTileExtensions to show off that all of this can (and should) be done with XAML

No worries, we'll continue to pursue creating a POCO editor for notifications! We want to provide the best dev experience.

Additionally, I would also be willing to further assist here in a capacity that improves everything but WinJS. Seriously... is anyone even using WinJS? ;)

Haha. Maybe the only solution would be forking the two projects, and maintaining a separate WinJS one. But I'm trying to avoid that.

I can see WWA (Windows Web Apps) becoming super popular with Hosted Web Apps for Windows 10. But with that, I'm not even sure they'd be able to take advantage of a WinRT library, meaning that my library would be useless to them? So yeah, choosing to support WWA was more of a "do this for completeness" decision, but it'd be interesting to find out if any devs are actually using WWA and using my library.

[Mike-E-angelo]

But the fact is, the XML came first, and the XML is still the truth, and the XML is the only thing natively supported in the platform, and because of that I feel the library needs to maintain a decent amount of consistency with the XML.

No argument there, but make sure it's properly-used XML and XML serialization. That means ideally serializing an object wholly as XML and then running it through XSLT, rather than hobbling and piecemealing it together as it is now. Again there are constraints here so I am aware that my ideals are exactly that. 😛 As long as everyone is aware of those ideals, then that is what is important to me (and others wanting clean code LOL).

I can see WWA (Windows Web Apps) becoming super popular with Hosted Web Apps for Windows 10.

I want some of what you're smoking and drinking then. :) Why would a web developer with HTML/JS skillz want to use (read: waste) their talents to make an application in a hosted environment that gets a fraction of exposure/accessibility that is abundantly available on the ubiquitous web? I think the terrible adoption of WinJS is very much an indicator of developers asking this very question. Plus, web developers do not have to pay $19 in registration fees (or 30% of in-app revenues) to build an application on the web; it's free. These are a few of the VERY MANY reasons why WinJS is ALL sorts of messed up. 😛 But the WORST of all is that the rest of us have to suffer greatly because of it as our design decisions have to account for a VERY backwards (and as you point out INCOMPATIBLE) way of thinking.

So, remind me again why Silverlight was such a bad idea? Haha, I kid, I kid... sort of. 😄

But seriously, all the WinJS projects I know were done because they were paid by MSFT to do. I don't know a single WinJS developer, and I would like to know who they are and why they develop in the face of the reasons above.

I will say that I don't blame you for accepting it for completeness. Mad props and respect, THAT is a challenge!! Now I have the opportunity to complain and discuss the reasons with you about why it's such a terrible idea!! 😆

[andrewleader]

You haven't looked up what Hosted Web Apps are, have you? They're new in Windows 10.

Basically, they let you submit your existing website, with its existing URL, to the app store, and it becomes an app. All your code remains on your server. You don't need to spend any extra time writing Windows-specific code. Your website runs just like it would in a browser, except it's an app now.

And if the devs want to integrate additional features like live tiles or toasts to their website, they can do that easily, by simply adding it to their server's JavaScript. They don't need to maintain two code bases. They do this: https://gist.github.com/Gr8Gatsby/3d471150e5b317eb1813#file-windows-ui-notifications-js

Do you think that's such a terrible idea? I think that's exactly what a lot of web developers are asking for. They want their website to be an app. They want to maintain the benefits of having a website, while also having an entry in the app store, and allowing people to launch their website "like an app", and some might even choose to integrate with platform-specific features like share targets and toast notifications. I think hosted web apps are a great idea, and nah I'm not drinking any Kool-Aid lol.