A closer look at Windows PowerShell OneGet - Part 2

In part one, we introduced the OneGet architecture and touched on providers, the components responsible for doing all the heavy lifting. Before we can dive into providers, however, we need to talk about OneGet plugins.

The OneGet Core API supports a simple plugin model that prevents coupling by way of Duck typing. That is, plugins don't reference any OneGet specific types or bind to any assemblies on disk. Instead, they implement a lightweight container encapsulating a few methods and properties that will be called and discovered at runtime, ultimately exposing any providers hosted within.

To get a better handle on what that means, let's look at a simple plugin hosting a single package provider:

public class Plugin
{
private PackageProvider _packageProvider;

public void InitPlugin(Func<string,
IEnumerable<object>, object> callback)
{
this._packageProvider = new PackageProvider();
}

public string PluginName
{
get
{
return "Sample";
}
}

public IEnumerable<string> PackageProviderNames
{
get
{
yield return "Sample";
}
}

public object CreatePackageProvider(string name)
{
return this._packageProvider;
}

public void DisposePlugin()
{
}
}
  • InitPlugin offers an opportunity to handle any internal initialization required. In this case, we create an instance of our single package provider class. We'll cover that class and that odd-looking callback parameter in a bit.

  • PluginName is used for simple book-keeping and logging.

  • PackageProviderNames lets OneGet know of any providers hosted by this plugin. In this case, we return a single provider named "Sample".

  • CreatePackageProvider is called once per provider listed by PackageProviderNames. These calls provide an opportunity to create package providers on-demand if needed.

  • DisposePlugin offers an opportunity to close or release any resources before plugin tear down.

Package providers bear the brunt of OneGet work and the required API surface reflects that. With the preview bits, providers are expected to implement about 14 methods.

Roll up your sleeves, here we go:

using Callback = Func<string, IEnumerable<object>, object>;

public class PackageProvider
{
public void AddPackageSource(string name, string location,
bool trusted, Callback c)
{
}

public bool FindPackage(string name, string requiredVersion,
string minimumVersion, string maximumVersion, Callback c)
{
return false;
}

internal bool FindPackageByFile(string filePath, Callback c)
{
return false;
}

public void GetInstallationOptionDefinitions(Callback c)
{
}

public bool GetInstalledPackages(string name, Callback c)
{
return false;
}

public void GetMetadataDefinitions(Callback c)
{
}

public bool GetPackageSources(Callback c)
{
return false;
}

public string GetProviderName(Callback c)
{
return "";
}

public bool InstallPackageByFastpath(string fastPath, Callback c)
{
return false;
}

public bool InstallPackageByFile(string filePath, Callback c)
{
return false;
}

public bool IsTrustedPackageSource(string packageSource, Callback c)
{
return false;
}

public bool IsValidPackageSource(string packageSource, Callback c)
{
return false;
}

public void RemovePackageSource(string name, Callback c)
{
}

public bool UninstallPackage(string fastPath, Callback c)
{
return false;
}
}
  • GetProviderName is simply a unique name for the provider.
  • FindPackage and FindPackageByFile handle the discovery of packages using the provider's own sources. A special fast path package identifier is returned for each package found for use during installation (see InstallPackageByFastpath).
  • InstallPackageByFastpath and InstallPackageByFile handle package installation using either a special fast path package identifier or by a manually specified file path. UninstallPackage simply removes packages by fast path.
  • AddPackageSource, GetPackageSources, GetPackageSources, and RemovePackageSource allow for package source management. Generally speaking, a package source is a location where packages can be found, identified by a Uri (e.g. http://my.host.com/packages). These locations can be flagged as trusted, skipping installation time confirmations.
  • IsTrustedPackageSource handles checking if a specified source is trusted or not. It's up to the provider to define trust.
  • IsValidPackageSource handles checking if a specified source is valid. It's up to the provider to define validity. This is used in the case where the host is given a specific package source for package installation. IsValidPackageSource will potentially identify the source as invalid, saving an expensive call to FindPackage.
  • GetInstallationOptionDefinitions and GetMetadataDefinitions allow for the specification of additional installation switches that bubble up to the host.

So let's talk about this Callback object that's passed into every method.

To allow plugins to call back into OneGet core API without a tight coupling between components (i.e. duck typing), plugins are offered a callback object. You can think of this object as a telephone handset on a very long cable, passed to each provider method, allowing the provider to call back into OneGet and talk to its core APIs.

Under the covers, the Callback object is simply a fancy delegate of type Func[string FunctionName, IEnumerable[object] Parameters, object Return]. (I had to use square brackets to avoid a bug in Squarespace at this time.) It's mainly used to retrieve delegates pointing to OneGet core APIs.

For example:

  • Callback(FunctionName, null) returns a Delegate to FunctionName if the function exists.
  • Callback(FunctionName, Parameters) actually invokes FunctionName. This pattern could be used to assert that FunctionName should always exist.

But rather than have a bunch of API resolution gunk in each provider method, I recommend you move that responsibility to a dispatcher of sorts.

Here's an example of a simple implementation:

public class Dispatcher
{
private Func<string, IEnumerable<object>, object> _callback;

public Dispatcher(
Func<string, IEnumerable<object>, object> callback)
{
if (callback == null)
throw new ArgumentNullException("callback");

_callback = callback;
}

private T GetRemoteFunction<T>() where T : class
{
var type = typeof(T);

var genericDelegate =
(Delegate)_callback(
type.Name.Remove(type.Name.Length - 2), null);

return Delegate.CreateDelegate(type,
genericDelegate.Target, genericDelegate.Method, true) as T;
}

// [...]

private delegate void AskPermissionFn(String permission);
public void AskPermission(String permission)
{
var fn = this.GetRemoteFunction<AskPermissionFn>();
fn(permission);
}

// [...]
}

This Dispatcher implementation could then be leveraged in a provider implementation as such:

// [...]

public void RemovePackageSource(string name, Callback c)
{
var dispatcher = new Dispatcher(c);
dispatcher.AskPermission("Are you sure?");

// [...]
}

In Part 3, we'll complete the provider implementation and cover some debugging tips/gotchas.

Feel free to stop by and say hello at one of the OneGet Community Meetings that happen every Friday at 10:00AM PDT over Lync. Everyone is welcome.

What's with the Rs, Bs, WRs, CCs, DCAs, and PTs in the Windows Store XML?

Like Raymond Chen, I spelunk around Windows during my lunch breaks. Today, I came across raw Windows Store XML -- the XML sent down to the Windows Store app on your Windows machine. I noticed immediately that it wasn't your ordinary semantic XML. Instead, the elements had been transformed into shortened, mechanical, almost alien versions, some comprising of only one letter.

Here's a snippet of XML delivered as part of the Where's My Water? 2 page:

<Emr>
<Pt>
<I>12c9d9c4-1b2a-4c79-8e36-5adad7c4d413</I>
<R>360a98aa-87f5-4e80-bda8-01880c2dbd38</R>
<B>360a98aa-87f5-4e80-bda8-01880c2dbd38</B>
<Pfn>Disney.WheresMyWater2_6rarf9sa4v8jt</Pfn>
<L>en-US</L>
<T>Where&#x2019;s My Water? 2</T>
<Wr>12</Wr>
<Ico>360a98aa-87f5-4e80-bda8-01880c2dbd38/Icon.287841.png</Ico>
<!-- // Snipped // -->
</Pt>
<!-- // Snipped // -->
</Emr>

Why would anyone name their elements this way?

I suspect this is an optimization to minimize the amount of data streaming from the datacenters hosting the Windows Store. (Or maybe storage on disk?) To compute a rough savings figure here, I copied the XML and replaced all the cryptic element names with saner versions. I then ran both through Visual Studio's document formatting feature (i.e. XML tidy) and gzip'ed them.

Here's a snippet of the cleaned up version:

<Entry>
<Product>
<ID>12c9d9c4-1b2a-4c79-8e36-5adad7c4d413</ID>
<ReferenceID>360a98aa-87f5-4e80-bda8-01880c2dbd38</ReferenceID>
<BaseID>360a98aa-87f5-4e80-bda8-01880c2dbd38</BaseID>
<PackageFamilyName>Disney.WheresMyWater2_6rarfa4v8jt</PackageFamilyName>
<Locale>en-US</Locale>
<Title>Where&#x2019;s My Water? 2</Title>
<Age>12</Age>
<Icon>360a98aa-87f5-4e80-bda8-01880c2dbd38/Icon.287841.png</Icon>
<!-- // Snipped // -->
</Product>
<!-- // Snipped // -->
</Entry>

Here are the compression results (using 7-Zip 9.32a 64-bit using a gzip ultra 32KB dictionary and 128 word size):

Original XML: 6,523 bytes
Renamed XML: 8,843 bytes
Difference: 2320 bytes

Gzip'ed original XML: 2,429 bytes
Gzip'ed renamed XML: 2,727 bytes
Difference: 298 bytes

As you can see, the difference between the gzip'ed original and modified XML markup is negligible. Even accounting for 110 million Windows 8 users, we're talking a relatively small savings of ~32.8GB.

Is it worth the extra transformation work on each side? (Think CPU, battery)

Maybe.

I took another look at the requests and replies going back and forth and it turns out, the Windows Store app omits the HTTP Accept-Encoding header that would normally indicate support for gzip'ed content. So it doesn't appear the XML is getting gzip'ed at all. That changes our savings figure to ~255.2GB.

Weird stuff.

A closer look at Windows PowerShell OneGet - Part 1

On Thursday, Distinguished Engineer Jeffrey Snover announced the availability of the Windows Management Framework 5.0 Preview. In tow was Windows PowerShell OneGet, touted as the new way to discover and install software on Windows. But after digging deeper into the preview bits, it’s clear OneGet is much more than a collection of cmdlets.

Let’s take a closer look.

OneGet_Figure01.png

Windows PowerShell OneGet is a bit of a misnomer -- it isn't strictly tied to PowerShell. Instead, OneGet represents a framework of hosts, providers and packages with a core set of APIs at its center. Peeking behind the curtains, we have three main component areas:

The OneGet core API houses all the package discoverability and installation support goop that powers OneGet hosts and providers. Specifically goop in areas such as asynchrony and operating system integration.

OneGet hosts tap into the core API. Typically, their job is to bridge the divide between the user and the API. Windows PowerShell, for example, exposes core functionality by way of handy cmdlets.

Requests from these hosts are relayed by the core to OneGet providers that are responsible for processing requests. The Chocolatey provider, for example, may receive a request for a package it does not have in its local cache. It will hold onto that request and retrieve the information it needs from its Internet-based package repository before responding.

To make that clearer, we'll walk through a simple PowerShell example.

OneGet_Figure02.png
  1. A script or user imports the Windows PowerShell OneGet module, tapping into the OneGet Core API. Install-Package is invoked to find and install the Notepad2 package.
     
  2. The Windows PowerShell host (hosting the OneGet module) makes a "find package" request via the OneGet Core API.
     
  3. The OneGet Core API locates a provider able to handle the request and passes it on.
     
  4. The provider, Chocolatey in this case, receives the request and scurries about, looking for the Notepad2 package. It downloads it from the central Chocolatey repository and proceeds to install it, providing progress and other information along the way. If all goes well, Notepad2 is now installed.

In Part 2, we'll take a closer look at how providers are implemented and try to hack one together.

Coming soon in Windows 8.1 Update 1: Internet Explorer Enterprise Mode

enterprise_mode.png

So, a build of Windows 8.1 Update 1 leaked onto the Internet last night. Buried in this build are early bits of what's being called Internet Explorer Enterprise Mode, a feature designed to "improve the compatibility of websites identified by your company".

There's not a lot to play with yet, but I tinkered with it a bit.

It appears to work similarly to the Compatibility View feature first introduced in Internet Explorer 8. One difference is that Enterprise Mode can be configured to consult a list of Enterprise Mode enabled sites via a file on disk or somewhere on the network. This file could contain a list of intranet resources, for example, that need to be rendered in a vintage document mode like IE8.

(Update -- Turns out, I missed Mary Jo Foley's write up on this feature in late January. Oops.)

There were also hints suggesting this mode would have the ability to run in its own sandbox, similar to InPrivate Browsing -- perhaps, and I'm guessing/hoping here, to allow for limited execution of older plugins and awfulware such as Java.

Neat.

You can turn on a limited version of this feature via a tweak to the registry where the F12 Developer Tools hangs out. Simply open the Registry Editor and navigate to the key HKLM\Software\Policies\Microsoft\Internet Explorer\Main. Add a DWORD value of EnableEnterpriseModeMenuOption with its data set to 1. Finally, mash the F12 key in a new IE window instance and you should see a enterprise new browser profile in the Emulation area.

emulation.png


Disable Skype integration in Outlook.com and SkyDrive

skypecall_cropped.png

A little over nine months ago, Microsoft rolled out Skype for Outlook.com. This new integration allows users to instant message, make phone calls and even send and receive video -- right from the web browser. It sounds great, but as my colleague Paul Thurrott pointed out then, it comes with a crippling incoming call notification that won't stop ringing after you answer the call with one of the many other applications or devices ringing.

I thought abandoning Outlook.com would be the cure, but Microsoft rolled this integration into SkyDrive too.

Determined to fix this, I ran Outlook.com through Fiddler and noticed Skype code was pulled down and initialized asynchronously from scripts hosted on skypewebexperience.live.com. I thought of various solutions involving DOM manipulation and code injection to unhook Skype, but opted for a simpler brute-force approach: I just pointed the sub-domain to 127.0.0.1 via the HOSTS file (effectively blocking it).

And aside from some minor scripting errors that appear in the browser console, it worked. It even worked in SkyDrive.

Good riddance.

Step-by-step instructions:

  1. Open Notepad as an administrator
  2. Open the file \Windows\System32\Drivers\Etc\Hosts
  3. At the end of the file, add 127.0.0.1 skypewebexperience.live.com and save.
  4. Close Notepad and restart SkyDrive/Outlook.com browser instances.