Linden Lab has announced that it is working on three projects that each will reduce some amount of unnecessary lag and delays from Second Life.
All three are positive improvements, and at least one has been on many peoples’ wishlists for rather a while now. Those aren’t the only things in the works, however.
The first (and perhaps the most radical) of the projects is to shift texture baking out of the viewer, and process it through a new farm of dedicated servers. This should speed up appearance changes, and significantly reduce the amount of data sent to Second Life viewers when appearance and clothing changes occur. This project has been codenamed Project Sunshine.
The second involves more modern and reliable HTTP communications through a new code library – though it isn’t clear whether that is one that has been developed in-house, or whether it is a third-party component.
The last, and perhaps most commonly requested over the years, is an improved system of object caching and cache-coherence signalling which will allow the viewer to take more advantage of data which it has already downloaded. That’s definitely long-overdue. My understanding is that proposals for all of these projects have been floating around the Lab for some considerable amount of time.
In addition, the Lab is shutting down one of its three data-centres, moving operations to just two, to minimise communications delays between centres, and eliminate two points of failure. The Lab is also shelling out a pretty-penny upgrading server hardware throughout its operation, in order to boost region-performance.
Collectively, the whole operation to reduce Second Life lag is codenamed Project Shining, and I expect pieces of it to roll out over the next six months, probably by the end of the year, all things being equal. These projects are big-deal items that must have taken considerable planning and pre-production before the Lab would even consider announcing them. It looks like the Lab is pretty serious about this. Let’s hope that it is successful, as well.
Linden Lab was not immediately available for additional comment.
YIPPEEEE!
Project Shining eh… They don’t read much Stephen King out there on the Bay, do they?
Interesting…
Distributing your assets throughout multiple datacenters makes sense if you’re taking advantage of global load balancing, replication disaster planning, etc. So if they’re still homing regions and various functions in specific server clusters without having fail-over, yeah, that’s a good move. Reduces number of locations you gotta monitor.
And the use of caching for… well… caching… that one earns an “about damned time” and a laugh.
Wondering if the pre-render has anything to do with nVidia’s rendering farm technology getting out there.
And I’m hoping the hardware outlays are meant to boost processing power and not cashcow replace old inefficient fully amortized assets with penny-pinching oversold virtualized server assets. (Region Idling still has the hairs on the back of my neck raised.)
Cautiously optimistic.
-ls/cm
I hear you re. hardware. Could mean better servers for us all. Could mean more regions crowded onto fewer servers (an economic win for LL, but of dubious value to us). Time will tell.
Interesting news, I hope this does pay off. I especially like how that blog entry has no actual name attached to it just “by Linden Lab”.
Well it sounds like someone at the Lab is finally getting there heads on straight. Too bad it’s about 8 years too late. But I see this as a positive step all the way around. Certainly these three things have needed attention for a very long time. I don’t know if “the right things too late” will be enough to save them. If not for the “deep pockets” and “merchants who have no choice” customers who still support LL’s hyper-expensive tier, likely LL would have been dead long ago. So whatever they can do now to make their existing customers happy is what they need to do; at their prices and the now-existing competition, it’s unlikely they’ll be seeing new ones any time soon.
So yes, this makes sense… and should have made sense to them years ago.
Tend to agree. Many things should have been done years ago. Not to belittle current progress, it’s good in theory.
The “look at how much we’re spending” bit can go. Costs offset for the one by nixing the other. If they’re going to publish actual financials it might mean something.
But right, a 3D world with too little pre-rendering, mesh with convoluted measurements, restrictions and only partially implemented, overpriced product, half finished projects and “won’t touch” bugs … it’s a long haul to get SL out of 2004.
This investment sends a strong signal that Linden Lab intends to remain lit up for a long time to come. /me likes this very much.
Just today I wondered if there were still any Lindens working on SL, since nobody ever answers to any Jiras, almost no Linden came to SL9B and there are no blog updates anymore. So, now I wonder if these new projects are somehow related to the sudden appearance of the new Second Life competitor Cloud Party.
From where I’m sitting, Harold, the Lab’s done more work on SL in the last 12 months than in any year previously. The problem is that it has become much harder to tell what it is working on, because it isn’t communicating.
Apparently, because of infrastructure changes, there’s new code being used in sim-code rollout process, and that’s why things went incredibly slow for the past couple of weeks: about three times as long, if not more.
Part of the problem seem to be the downtime per region: I’ve seen regions down for over an hour.
I don’t know if testing this sort of thing on the Beta grid is so very useful, because of size differences and how the multiple data centres are used, but the difference is so huge, it’s a bit hard to explain why the Lindens haven’t said anything through formal channels. If it doesn’t improve, there’s a couple of days a week that are going to be bad for those of us in Europe.
Everyone wants ‘formal’ channels. The formal channels are simply not the place for the day-to-day goings on. We get that in the user groups. The Shining announcement is something for the formal channel. We know about most parts of the announcement from UG meetings. The announcement only put things in a context.
I doubt their PR people understand the technical side of SL well enough to understand what needs to be conveyed. The techies don’t have the time and if they did I’m not sure how well they could explain things to the SL users.
Another significant problem is user behavior. The Lindens must dread talking to the community in general. No matter what they say there are enough rude and abusive wackadooles on any side of an issue to make communication miserable. There is no way for the Lab to win. I’m not surprised they limit communication.
Any one that wants the Lab to do a better job of communicating, needs to get the community to do a better job of communicating first.
Lots of people don’t put much effort in communicating with the Lab ’cause the Lab mistreated them in the past; the Lab needs to re-earn the respect it lost, and the way up is much harder than the way down.
And the relationship is not symmetrical, way more is expected from a company than is from a random customer.
Totally agree that we as customers have to communicate better as well, but Linden Lab has to be receptive of communication in better ways than JIRA and user group meetings held during their work hours (incidentally, other people’s work hours). I know anyone can add a user group item even if they don’t attend, but editing a wiki and parsing a chat log a week later isn’t compelling enough a medium to express concern and get feedback on it. Linden Lab could really use one of those UserVoice things.
“Formal channels” are those that we can all access. I don’t want to have to spend time and bandwidth on attending a UG meeting which can only ever be attended by a few dozen people. And Linden Lab should not depend on people such as yourself to tell the rest of us.
But this general issue of careless, ineffective, communications, is something I am inclined to get ratty on. In some ways, my personal situation is akin to Tateru’s, and I have experienced too many instances of bad communication by healthcare so-called professionals. Linden Labs isn’t likely to kill anyone with their mistakes. That seems to be their only advantage.
Another big project, hooray!
Does this mean they have finished something?
I was wondering when the not-so-new-anymore Bagman Linden would make good on his self-description of:
“I describe myself primarily as a hard-core C++ developer, with a passion for well-architected, highly optimized systems. I like taking on the big challenges and enjoy refactoring systems to make substantial improvements, rather than just make smaller, incremental improvements.”
I guess rightfully it takes a long time to update such a big legacy system without accidentally breaking things. Good luck to them in the months ahead and I’ll deal with any accidental downtime so long as its in the pursuit of making SL a lot better in the long run.
Yay! but it should be called Project Ooh, Shiny!
Bender: Ah-ah-ah! What do we say when someone gives you something?
Kid: ‘BOUT TIME!
Futurama S3 Ep9: The Cyber House Rules
Nothing like some competition to get LL’s eye on the ball again.
Sim-crossings are already better in vehicles, I’m pleased to say. The dream of a road-rally lives on…
If you’re thinking of timing, this had to have all started long before we’d ever heard of (say) Cloud Party. Being that it is a strategy announcement on the last working-day of the quarter, I’d say even the timing of the announcement didn’t involve external influences.
Clearly they could not, in the last week, have rolled out the region-crossing improvements I’ve seen. And other improvements needed time and lots of coding.
Yet timing is everything. I am not sure I agree with you on that issue. Releasing this news while Cloud Party is gaining logins from the press at NWN seems rather savvy marketing…wait.
It’s Linden Lab we are discussing. For them, “savvy” and “marketing” don’t seem to go in the same sentence.
My stats indicate that the servers have been consistently terrible: with a only 1.3% of all sims running at 45 FPS, and a huge number of them running at far slower speeds.
I just ran an analysis of Time Dilation (TD) and Frames Per Second (FPS) for 25,379 active sims recently visited by a very low-lag Metabolt (non GUI) client, and the results are awful.
See the graph at http://metaverse.mitsi.com/stat.jpg
The system running the tests is a low ping time, FIOS fiber-optic, 8 core I-7 with flash drives and 24 Gigs of RAM, so its not the client causing this! The Linden server reports these figures.
Indeed, anything under 40fps (at the simulation-side) is a cause for concern. Nice graph, by the way, and your collection methodology is sound.
Declaring war on lag is good.
Killing scripts by idleing the sim when nobody is around is bad.
http://www.sluniverse.com/php/vb/scripting/74068-how-do-i-run-bot.html
People are going to start running bots everywhere just to keep their meeroos breeding and timer events running accurately. I can see this snowballing very quickly if LL doesn’t roll this ‘feature’ back.
Now now Bubblesort, think of all the advantages:
* Free rental time. If rental scripts aren’t running, they can’t track rent, right?
* None of that pesky script failure when we’re offline. If they’re not running, they can’t fail, right?
* Oh oh… can we extend this concept to no tier when no one is on the sim?
I’ve had hosts idle always-on software of mine before…but only at free tiers of service to prevent exploitation. I’ve never paid anyone 300 dollars a month for anything that they just automatically ‘idle’.
Water idling! Oh your lawn is dead because youe sprinklers barely worked? You should’ve been home. Also we’re charging you as if you used 100% of the water you usually do instead of the 10% you actually used.
Electricity idling! Your gold fish are dead? You should’ve been home. Oh and we’re going to charge you as if your aquarium pumps were working anyway.
Blog idling! Let’s 404 those RSS feeds since there hasn’t actually been a person on the site itself the last 5 minutes. Oh, and let’s charge the same.
A big part of Second Life’s economy is built on hacks and paying others to get around or finish features Linden Lab puts out. Not really surprising region idling comes half-baked.
I am all for these, especially the caching, but I rarely change clothes and could care less how long it takes, so long as it actually works! We all know how well the current baking works, right?
Region Idling has issues but in general, I would support it if it would help for the horrid conditions most sims have been in for years. As Bubblesort’s link points out, region idling easy to get around with a simple script.
I just went to 9,920 sims, and only 18% were performing at full speed with a TD of 1.00 , 52% were running at 90% to 100% speed, and 16% were at worse than 50% Time Dilation. The only good news is that half the sims run at 43 FPS or better, the bad news is the rest are slower, in some cases (159 sims) running at less than 10 FPS!
You have to admit these are pretty awful figures, and what is worrying is that none of the above fixes from the Lindens will have any effect on this. And Region idling has nothing to do with it, as an avatar was in the sim at the time!
The problem is that simple script has to ping an HTTP server every 5 seconds or else the region idles. It’s going to create a lot of lag if every script starts doing that constantly because no scripter wants to rely on somebody else to make sure their timers work right. The short delay means that we’ll have to be pinging HTTP constantly. This method will also start eating up the available URLs in a region if all scripts start using a canned function that grabs a URL and pings itself every 5 seconds.
The alternative is to start running bots everywhere. This is attractive to consumers because that is one way to make absolutely certain the region will not idle.
In practice, people will not be choosing one fix over the other. Scripters will script as if the customer will not be running a bot and customers will run bots as if the scripters do not know how to fix the issues. This means that everybody will be applying multiple fixes at once, with each fix eating more resources needlessly because LL is incompetent.
I’m awarding the first round in the war on lag to the lag. LL needs to step up it’s game if it’s going to make this a serious fight.
I AGREE!!! ONE SIM FOR EACH SERVER that will fix the lag! FOR THE AMOUNT OF THE MONEY THEY CHARGE FOR A SIM can’t LL do this??? I Don’t worry about stupid clothes– stop the lag!!! Don’t pretend that it is the user.- Ferd you are smart-
I wonder if this is really secretly named “Project: Tier Raise” to justify the need to raise rates in the future to make up for the slow bleed off of estate regions.
It’s great they are “shelling out a pretty-penny upgrading server hardware throughout its operation, in order to boost region-performance.”, which will probably mean even more sims stuffed into a box than before, but leaves me wondering who will end up paying for that investment. Surely it will be the customers.
I will believe all this when I see it.