Technology - Google News

Google, Facebook, Microsoft, and Twitter partner for ambitious new data project
The Fortnite Team Talks Matchmaking Drama in Playground LTM Postmortem
Nintendo Drops Free Breath of the Wild DLC in Mario Kart 8 Deluxe

Google, Facebook, Microsoft, and Twitter partner for ambitious new data project

Posted: 20 Jul 2018 03:00 AM PDT

Today, Google, Facebook, Microsoft, and Twitter joined to announce a new standards initiative called the Data Transfer Project, designed as a new way to move data between platforms. In a blog post, Google described the project as letting users "transfer data directly from one service to another, without needing to download and re-upload it."

The current version of the system supports data transfer for photos, mail, contacts, calendars, and tasks, drawing from publicly available APIs from Google, Microsoft, Twitter, Flickr, Instagram, Remember the Milk, and SmugMug. Many of those transfers could already be accomplished through other means, but participants hope the project will grow into a more robust and flexible alternative to conventional APIs. In its own blog post, Microsoft called for more companies to sign onto the effort, adding that "portability and interoperability are central to cloud innovation and competition."

The existing code for the project is available open-source on GitHub, along with a white paper describing its scope. Much of the codebase consists of "adapters" that can translate proprietary APIs into an interoperable transfer, making Instagram data workable for Flickr and vice versa. Between those adapters, engineers have also built a system to encrypt the data in transit, issuing forward-secret keys for each transaction. Notably, that system is focused on one-time transfers rather than the continuous interoperability enabled by many APIs.

"The future of portability will need to be more inclusive, flexible, and open," reads the white paper. "Our hope for this project is that it will enable a connection between any two public-facing product interfaces for importing and exporting data directly."

The bulk of the coding so far has been done by Google and Microsoft engineers who have long been tinkering with the idea of a more robust data transfer system. According to Greg Fair, product manager for Google Takeout, the idea arose from a frustration with the available options for managing data after it's downloaded. Without a clear way to import that same data to a different service, tools like Takeout were only solving half the problem.

"When people have data, they want to be able to move it from one product to another, and they can't," says Fair. "It's a problem that we can't really solve alone."

Most platforms already offer some kind of data-download tool, but those tools rarely connect with other services. Europe's new GDPR legislation requires tools to provide all available data on a given user, which means it's far more comprehensive than what you'd get from an API. Along with emails or photos, you'll find thornier data like location history and facial recognition profiles that many users don't even realize are being collected. There are a few projects trying to make use of that data — most notably Digi.me, which is building an entire app ecosystem around it — but for the most part, it ends up sitting on users' hard drives. Download tools are presented as proof that users really do own their data, but owning your data and using it have turned into completely different things.

The project was envisioned as an open-source standard, and many of the engineers involved say a broader shift in governance will be necessary if the standard is successful. "In the long term, we want there to be a consortium of industry leaders, consumer groups, government groups," says Fair. "But until we have a reasonable critical mass, it's not an interesting conversation."

This is a delicate time for a data-sharing project. Facebook's API was at the center of the Cambridge Analytica scandal, and the industry is still feeling out exactly how much users should be trusted with their own data. Google has struggled with its own API scandal, facing outcry over third-party email apps mishandling Gmail users' data. In some ways, the proposed consortium would be a way to manage that risk, spreading the responsibility out among more groups.

Still, the specter of Cambridge Analytica puts a real limit on how much data companies are willing to share. When I asked about the data privacy implications of the new project, Facebook emphasized the importance of maintaining API-level controls.

"We always want to think about user data protection first," says David Baser, who works on Facebook's data download product. "One of the things that's nice about an API is that, as the data provider, we have the ability to turn off the pipeline or impose conditions on how they can use it. With a data download tool, the data leaves our hands, and it's truly out there in the wild. If someone wants to use that data for bad purposes, Facebook truly cannot do anything about it."

At the same time, tech companies are facing more aggressive antitrust concerns than ever before, many of them centering on data access. The biggest tech companies have few competitors. And as they face new questions about federal regulation and monopoly power, sharing data could be one of the least painful ways to rein themselves in.

It's an unlikely remedy for companies that are reeling from data privacy scandals, but it's one that outsiders like Open Technology Institute director Kevin Bankston have been pushing as more important than ever, particularly for Facebook. "My primary goal has been to make sure that the value of openness doesn't get forgotten," Bankston says. "If you're concerned about the power of these platforms, portability is a way to balance that out."

Update 7/20/2018 12:00PM EST: This piece was updated to include reference to Microsoft's announcement of the Data Transfer Project.

Let's block ads! (Why?)

The Fortnite Team Talks Matchmaking Drama in Playground LTM Postmortem

Posted: 20 Jul 2018 07:49 AM PDT

Fortnite Server Shutdown

The Fortnite Team takes some time to reflect on what went wrong when Playground Mode first released, what it taught them, and how they fixed it. Primarily, the issue was based on how matchmaking functions in Fortnite. The Matchmaking Service (MMS) relies on a series of nodes. Within each node is a list of servers that can work. The MMS assigns players to a node, which in turn selects a server. Fortnite‘s standard Battle Royale mode involves 100 players, while Playground Mode involves one to four. What it boils down to is the fact that more matches needed to be made, requiring 15 times as many servers. Playground Mode was so demanding on the system that it essentially caused the whole thing to “grind to a halt.”

The solution was to split off Playground Modes Matchmaking Service to “run on its own service cluster.” This prevented Playground Mode issues form affecting the main game while allowing the team to isolate and target the problem. They upgraded the MMS cluster so that it could “bulk rebalance sessions from other nodes to ensure repeated lookups were not necessary.” This is an improvement that will be brought over to the main MMS cluster as a way of “future-proof[ing]” the system.

Overall, the Fortnite team learned about the weak points within their matchmaking system and realized it’s better to roll these things out more slowly. As a result they “opted to bring back Playground in small steps by individual regions and platforms, with the goal of reducing the initial load on the system so [they] could scale into it.” In the end the team had this to say about the whole situation: “the process of getting Playground stable and in the hands of our players was tougher than we would have liked, but was a solid reminder that complex distributed systems fail in unpredictable ways.” Fortunately, this learning experience should carry the team into a better future as the platform continues to grow.

Below is the entire postmortem for those interested in all the technological details:

We recently stood up our Playground LTM on June 27th, at about 4 AM EDT. Following this, we experienced an overload of our matchmaking service which caused both the default modes and Playground to fall over. We worked to get the service to where it needed to be, and were finally able to roll out the mode on the evening of July 2nd.
What happened?
Our matchmaking is built on something called the Matchmaking Service (MMS), which is responsible for facilitating the “handshake” between players looking to join a match and an available dedicated server open to host that match. Each node in the matchmaking cluster keeps a large list of open dedicated servers that it can work with, randomly distributed by region to keep a roughly proportional amount of free servers for each. Players that connect to MMS request a server for their region, MMS assigns that player to a node, and the node picks a free server for the requested region from its list.
Since Playground mode makes matches for every 1-4 people instead of 100, it requires between 25 and 100 times as many matches as normal depending on party size. While we could pack virtual servers a bit tighter per physical CPU for Playground mode, we still had to use 15 times as many servers as we had been running for the other modes. We were able to secure the total server capacity, but it meant the list that each node had to manage was suddenly 15 times as long as well.
When an MMS node can’t find a free server for the requested region within its own list, it has to go ask all of the other nodes for a spare one by reading from each of their local lists. When you’re a node and your list is suddenly 15 times longer, it slows you down. When you have to go check all of the other lists and each one is also 15 times longer, it slows you down up to 15 times per node, which can translate to computation times that are orders of magnitude longer than normal. When we released Playground, the overwhelming demand quickly exhausted the local lists for MMS nodes far faster than the system could refresh them. Each node was running to every other node to request extra servers that just weren’t there yet, or at the very least took a long time to pick out of the non-local lists. The long compute times caused the CPU to end up with a backlog of pending requests, resulting in a feedback loop that eventually caused the system to grind to a halt.
What did we do to fix it?
The first thing we did after disabling the mode was to split Playground MMS to run on its own service cluster. This was necessary not only to keep a traffic jam from affecting the base game modes, but also to allow us to iterate and tweak the service as often as we needed while we worked to get Playground back online. We tried increasing levels of dramatic re-architecturing, and tested at each stage until we reached the acceptance criteria to re-release the mode.
Once we identified the root of the problem as the exhaustion of sessions from local lists, the solution was to give the cluster the ability to bulk rebalance sessions from other nodes to ensure repeated lookups were not necessary. With the system constantly shifting regional capacity from nodes with an excess to nodes that might be running low, the odds of a node running dry for a particular region and having to search outside its local list have been drastically reduced. While not an issue right now in the primary Fortnite Battle Royale game modes, this is an upgrade we are bringing over to the main MMS cluster as well to future-proof the system.
We pushed the load-testing process to the limits during our MMS restructuring, because the scale of what we were trying to simulate was so far beyond normal usage or testing patterns. We needed to spin up many millions of theoretical users and hurl them at our Playground MMS system in a big, crashing wave in an attempt to strain our new session rebalancer. While the tweak – test – evaluate cycle took several hours per loop, it allowed us to develop and refine the rebalance behavior to a point where we felt it could stand up to the traffic, as well as to identify and fix edge-case bugs that could have torpedoed the effort to bring Playground back online.
What have we learned?
In short, we learned a lot about our own matchmaking system and its failure points as well. We planned and prepared for what we thought to be the maximum sustained matchmaking throughput and capacity based on the size of our player base (plus a healthy buffer), but didn’t properly anticipate the edge-case of of the initial “land rush” of players exhausting local lists.
On the restart of the mode itself, we had an additional learning experience. We opted to bring back Playground in small steps by individual regions and platforms, with the goal of reducing the initial load on the system so we could scale into it. We actually encouraged the opposite, as players swapped regions into those that had the mode re-enabled and forced us to slow the rollout as we dealt with capacity issues. The silver lining is that we certainly have much better visibility into the total available cloud resources in Asia than ever before, and we want to give a shoutout to our cloud partners for working with us to ensure we could quickly adjust!
The process of getting Playground stable and in the hands of our players was tougher than we would have liked, but was a solid reminder that complex distributed systems fail in unpredictable ways. We were forced to make significant emergency upgrades to our Matchmaking Service, but these changes will serve the game well as we continue to grow and expand our player base into the future.

[Source: Epic Games]

Let's block ads! (Why?)

Nintendo Drops Free Breath of the Wild DLC in Mario Kart 8 Deluxe

Posted: 20 Jul 2018 07:45 AM PDT

People have been asking for it since Mario Kart 8 Deluxe was released on the Nintendo Switch, and now players have it as a new free DLC. Last night, Nintendo officially added a new cycle and character from The Legend Of Zelda: Breath of the Wild, specifically the Master Cycle Zero and Link wearing his Champion’s Tunic. The company tweeted out a trailer to show off the look as he races around the track.

A new update to #MarioKart 8 Deluxe is available now for #NintendoSwitch! Race like a champion with the Master Cycle Zero and Champion’s Tunic Link from The Legend of #Zelda: Breath of the Wild! This update also includes the Ancient Tires and the Paraglider parts! pic.twitter.com/cpxwRpdhcB

— Nintendo of America (@NintendoAmerica) July 20, 2018

We reported last week that there was more unexpected content coming to Mario Kart 8 Deluxe, but what exactly we’d be seeing was unclear. If it’s more versions of racers that are already in the game, that’s nice, but kind of weird considering we haven’t seen much added since the game was released. We would prefer to see new tracks and modes added since the roster is already stacked (unless they wanna toss some Super Smash Bros. people in the mix for once, like Captain Falcon, Metroid, and Pit.)

(Last Updated July 20, 2018 7:22 am )

Let's block ads! (Why?)