A defense of vgchartz - Page 2 - High-Def Digest Forums
Reply
 
Thread Tools Search this Thread Display Modes
  #11  
Old 11-08-2007, 08:36 PM
james's Avatar
Senior Member
 
Join Date: Dec 2006
Posts: 18,141
Default

You might as well start a thread about the defense of inaccurate speculation. It's almost the same thing.
Reply With Quote
  #12  
Old 11-08-2007, 09:40 PM
BStroms's Avatar
Senior Member
Thread Starter
 
Join Date: Jun 2007
Posts: 1,088
Default

Ok, I've given my opinions, I'd like those who are still against VGChartz to answer a couple of question for me if they don't mind.

1. Do you believe the people at VGChartz are telling the truth about getting actual sales data and using that to estimate total sales? If not, why do you assume they're lying?

2. Are you also against using NPD, media create, Famitsu, etc?

I've already shown a 29% difference in just the first week I compared them, and I highly doubt that's on the extreme end of the spectrum. I have no reason to believe our American NPD numbers are any more accurate than what the Japanese can come up with.

And to use Godstar's example, assume the two consoles sold evenly and NPD is off 29% in one direction on one, and 29% in the other direction with the other. It would show one system as outselling the other, 1.82 to 1. Surely that's unacceptable right? If it could be off by a lot than of course there's no value whatsoever in the data and we should ignore the person that's making the argument or ban people that used the numbers as some have suggested for vgchartz.
Reply With Quote
  #13  
Old 11-08-2007, 09:52 PM
RedRedSuit's Avatar
Senior Member
 
Join Date: Mar 2007
Posts: 25,777
Default

Quote:
Originally Posted by BStroms View Post
1. Do you believe the people at VGChartz are telling the truth about getting actual sales data and using that to estimate total sales? If not, why do you assume they're lying?
I don't know if they're telling the truth. However, the fact that reputable companies don't use them doesn't bode well for them. In addition, the fact that their site is full of popups and, at least formerly, spyware doesn't bode well for them, either.

Quote:
2. Are you also against using NPD, media create, Famitsu, etc?
Media Create and Famitsu? Yes. NPD, no. NPD are widely trusted by reputable companies.

Quote:
I have no reason to believe our American NPD numbers are any more accurate than what the Japanese can come up with.
Given that Sony, Microsoft, and Nintendo repeatedly quote NPD numbers leads me to believe that they know what they're talking about -- at least to a far greater extent than VGChartz, a spyware-ridden website.

Quote:
we should ignore the person that's making the argument or ban people that used the numbers as some have suggested for vgchartz.
In case you're talking about me, I said that VGChartz should be banned, not people citing them.
Reply With Quote
  #14  
Old 11-08-2007, 10:42 PM
BStroms's Avatar
Senior Member
Thread Starter
 
Join Date: Jun 2007
Posts: 1,088
Default

Quote:
Originally Posted by RedRedSuit View Post
In case you're talking about me, I said that VGChartz should be banned, not people citing them.
It's possible that's what I was thinking of, but in my defense, how do you ban vgchartz without banning the people who use it. Not much of a ban if you just say pretty please don't do this


And I do reiterate, why on earth would NPD care the slightest bit about vgchartz if there weren't some reputable companies that pay NPD money that are considering bailing because they can get data for free? They came off as far to angry for me to believe there wasn't something behind it, at the very least fear of what may come.

Finally, the fact that companies use and quote NPD doesn't mean that there numbers are good, only that they're the best available. There's quite a difference. They could still have a +- 50% margin of error, what do you really have to benchmark it against?
Reply With Quote
  #15  
Old 11-08-2007, 11:07 PM
RedRedSuit's Avatar
Senior Member
 
Join Date: Mar 2007
Posts: 25,777
Default

Quote:
Originally Posted by BStroms View Post
It's possible that's what I was thinking of, but in my defense, how do you ban vgchartz without banning the people who use it. Not much of a ban if you just say pretty please don't do this
Good point.

Quote:
And I do reiterate, why on earth would NPD care the slightest bit about vgchartz if there weren't some reputable companies that pay NPD money that are considering bailing because they can get data for free? They came off as far to angry for me to believe there wasn't something behind it, at the very least fear of what may come.
They probably don't like people, like bloggers, listing numbers they feel are poop. Consider it professional pride.

Quote:
Finally, the fact that companies use and quote NPD doesn't mean that there numbers are good, only that they're the best available. There's quite a difference. They could still have a +- 50% margin of error, what do you really have to benchmark it against?
I don't. It's a leap of faith. But I'd rather do a leap of faith for those numbers than some random spyware-ridden website's.

Last edited by RedRedSuit; 11-08-2007 at 11:40 PM.
Reply With Quote
  #16  
Old 11-09-2007, 02:36 PM
Gnome's Avatar
Senior Member
 
Join Date: Jan 2007
Posts: 5,051
Default

Quote:
Originally Posted by BStroms View Post
Many people on these forums have a dislike for vgchartz and some seem to downright hate it. It's not ever going to be 100% accurate but it's far from being the completely useless guesses many people make it out to be.
VGChartz is not accurate for predicting data until other data is released that they can steal and apply to their site. Therefore it's useful for looking at longterm trends over 6-12 months or more, and to quickly look up sales in the past but as far as a measure for anything under 1 month old it IS completely useless as more than a general ballpark figure of where sales are at.


Quote:
Originally Posted by BStroms View Post
Remember NPD numbers are not 100% accurate. They don't have all the numbers, they have some of them and use their formulas to estimate the total.
NPD is a professional marketing data company, they collect the data from 60% of the industry and have sophisticated models based on past sales to predict the relative marketshares their sources have to the overall sales. They are straight up the most accurate method one can use for finding sales data, even MS, Sony, and Nintendo use them in their official press releases.

Quote:
Originally Posted by BStroms View Post
The same method that vgchartz uses.
wrong, vgchartz collects from what they claim is 5% of the industry (it's suspected they collect data from one specific retail chain) and use that data to project the overall sales for Japan and the US. They then take those numbers and apply a sales model to determine what approximate European numbers are. Then when real data comes out they quickly adjust their numbers to be within 5-10% of the actual data, and write up a report about how accurate and close their numbers were.

Quote:
Originally Posted by BStroms View Post
Granted the NPD has more data, leading to more accurate numbers, they don't have a crystal ball that tells them the exact sell through figures like some people seem to assume.
NPD is the most accurate source of sales data available. To anyone. Manufacturers can give us shipped numbers if you like, but that doesn't represent anything because that only counts sold to retailers.

Quote:
Originally Posted by BStroms View Post
So how accurate are they? It's impossible to tell, as they have no real competitor on the same level as them.
No? NPD competes with them for NA, Media Create for Japan, and one could argue ChartTrack and GfK compete in Europe.

One could also argue that Neogaf and TheSimExchange compete because they both try to predict actual sales as well.

There are people on Neogaf's prediction leagues that get within 5% of NPD numbers, and TheSimExchange averages between 2-10% on any given month with it's numbers.

VGChartz numbers are between 5-25% off on any given month, depending on whether you're talking about hardware or software, they are way worse at predicting software.

Quote:
Originally Posted by BStroms View Post
But there is a country that does have two more professional services. Japan has both Famitsu and Media Create releasing numbers.
Famitsu releases Media Create's numbers, they aren't seperate numbers.

Quote:
Originally Posted by BStroms View Post
And they even release for the exact same time frame making comparison all the easier. So let's take a look at the most recent data I could find released by both. 10-22-07 through 10-28-07.

Famistsu
Wii 35000
NDS 83000
PS3 16000
PS2 11000
PSP 62600
360 3500

Media Create
Wii 27502
NDS 76243
PS3 18785
PS2 11698
PSP 59792
360 3718

Deviation
Wii 27%
NDS 9%
PS3 11%
PS2 6%
PSP 5%
360 4%

This gives an average deviation of 10.33%
Got links for this?

Quote:
Originally Posted by BStroms View Post
For the most recent vgchartz to NPD comparison you have to go back to the month of August
http://www.vgchartz.com/news/news.php?id=564
Here you get the following deviations
Wii 10%
NDS 13%
PS3 20%
PS2 20%
PSP 12%
360 11%

This gives an average deviation of 14.33%. If vgchartz is worthless with an average deviation of 14.33% then at least one of the 'official' numbers from japan must be highly unreliable at 10.33%. But looking at it reasonably, this is a field of estimating, where no one knows exactly what the right number is. And only 14.33% off on average is quite good. (And some of that will be due to NPD not being completely accurate.)
Are those deviations before or after they updated their numbers to reflect the NPD data? They were 80% off with their Bioshock sales numbers before they adjusted them.

Quote:
Originally Posted by BStroms View Post
Of course it's always possible that vgchartz is lying outright about their methods and have just gotten lucky lately in the fact that their hardware numbers have been getting closer and closer to NPD numbers all year long. Even their software numbers (which they admit themselves are less accurate) have been getting better. But the NPD's recent behavior has be thinking that's unlikely. Their rant against online sites that estimate the numbers and thier cutting back on publicly released data tells me something has them spooked.
Uh, NPD gets paid by people to find out what these numbers on. VGChartz takes those numbers and gives no credit to NPD and reposts them as their own. Wouldn't you get pissed too?

Quote:
Originally Posted by BStroms View Post
They don't care if you and I visit vgchartz and believe their numbers or not. We don't pay NPD any money.

The fact that they're upset tells me that some of their clients are starting to wonder why they should pay NPD when vgchartz gives out numbers for free which are getting more accurate. And reducing the amount of data that is public will make it harder for vgchartz to benchmark their numbers and update their formulas to give more accurate estimates.
Why should VGChartz benefit from NPD's hard work?

Quote:
Originally Posted by BStroms View Post
Conclusion: Yeah, don't hold vgchartz numbers as gospel, but then again don't do that with NPD either. On the other hand, their numbers seem pretty good, and will likely continue to get more accurate as they talk more businesses into giving them data, and refine their formulas. The hardware numbers especially are certainly good enough now, at least in the US, to treat as if those are the correct numbers until the NPD data is released. Especially since we'll be seeing less from the NPD in the future.
NPD numbers ARE Gospel. You can't get any more accurate data than them in the industry for US sales.
VGChartz rips those numbers off. They were directly stealing them for a while, then NPD threatened to sue them so they changed to doing weekly updates and no longer claim to use NPD numbers, even though they still DO use them.
VGChartz gets in the ballpark for Hardware, but their software predictions are total shit. There are more accurate sources for Hardware though. We had a couple people on here who used to swear by VGChartz numbers, and they were used as the basis of claims that the PS3 outsold the 360 a few months back. When we got real sales data we found out that wasn't the case. VGChartz numbers are OK for using when looking at something like sales over the last year, or even farther back, but they are NOT good for using as the basis of smackdown arguments here because unfortunately that margin for error is enough to make a difference at such a small time frame.
Reply With Quote
  #17  
Old 11-09-2007, 03:03 PM
Gnome's Avatar
Senior Member
 
Join Date: Jan 2007
Posts: 5,051
Default

Quote:
Originally Posted by BStroms View Post
They do go back and update totals, but they also do a comparison every single month of the numbers they came up with, and the NPD numbers. With all of their numbers officially released before the NPD numbers come out, and the number of people who hate that site, don't you think someone would have, you know, taken a screenshot of their numbers as they came out and caught them cheating when they did the comparison?

No one has, because they don't. So it doesn't invalidate any of the comparisons I've made.
When VGChartz is really far off they adjust their numbers to within 10% and then post that report.
Reply With Quote
  #18  
Old 11-09-2007, 03:04 PM
RedRedSuit's Avatar
Senior Member
 
Join Date: Mar 2007
Posts: 25,777
Default

Gnome has arrived to administer sales knowledge pwnage. Have you considered a career in this? You seem quite into it.
Reply With Quote
  #19  
Old 11-09-2007, 03:06 PM
Godstar's Avatar
Senior Member
 
Join Date: Jan 2007
Posts: 23,629
Default

Quote:
Originally Posted by RedRedSuit View Post
Gnome has arrived to administer sales knowledge pwnage. Have you considered a career in this? You seem quite into it.
I think he really is in that market to some degree.
Reply With Quote
  #20  
Old 11-09-2007, 03:50 PM
BStroms's Avatar
Senior Member
Thread Starter
 
Join Date: Jun 2007
Posts: 1,088
Default

Quote:
Originally Posted by Gnome View Post
NPD is a professional marketing data company, they collect the data from 60% of the industry and have sophisticated models based on past sales to predict the relative marketshares their sources have to the overall sales.
Just to go right back at you for a comment you make later in your reply, do you have a link to the 60%? I'm quite impressed if they managed to get such a high percentage of the sales data. I would have pegged it at 30-40%.

Quote:
wrong, vgchartz collects from what they claim is 5% of the industry (it's suspected they collect data from one specific retail chain) and use that data to project the overall sales for Japan and the US.
No actually it's right. I said they use the exact same method with the difference being scale and you say it's a different method because NPD gets 60% and estimates the other 40% and vgchartz gets 5% and estimates the other 95%? That is the same method. I hope whatever vgchartz % of the sales data is that can continue to get more, improve their methods, and become a real competitor to NPD that offers a lot more free data.

Quote:
They then take those numbers and apply a sales model to determine what approximate European numbers are. Then when real data comes out they quickly adjust their numbers to be within 5-10% of the actual data, and write up a report about how accurate and close their numbers were.
People keep making that accusation, and it's a pretty serious one. But also one that's very easy to prove if it's true. If you want to make a claim like that, get the screenshots and show them changing their numbers then claiming that the modified numbers were there original for comparison. Innocent until proven guilty right?


Quote:
NPD is the most accurate source of sales data available. To anyone. Manufacturers can give us shipped numbers if you like, but that doesn't represent anything because that only counts sold to retailers.
Exactly, it's the best available. If the exact numbers came out a couple weeks after NPD people would say to ignore the NPD as worthless until the exact numbers came out. And until NPD numbers come out, VGChartz is the best we have. Especially with NPD cutting back. Sure you can't say with conviction which console sold more when they're close, but even on the month that the PS3 lost to the 360 when VGChartz showed the 360 winning, I believe it was close enough to be within the margin of error for NPD.

At the very least it can be used as a tool to say why you think one system or another is doing well. If VGChartz shows the PS3 with a 500% increase from one week to the next with 360 sales plummeting week after week, then it would certainly be more likely that the PS3 would win that month.

Sure there's a lot of speculation and uncertainty as well, but that doesn't mean people shouldn't use it. Are you also against the use of release dates in posts? Those are highly speculative and can often end up being way off, even when the date was supposed to be very soon. So for people who want to ban VGChartz numbers do you want to ban all release dates being mentioned in an argument too until they're actually out?


Quote:
No? NPD competes with them for NA, Media Create for Japan, and one could argue ChartTrack and GfK compete in Europe.
You misread that, the NPD has no real competitors, not VGChartz.

Quote:
Famitsu releases Media Create's numbers, they aren't seperate numbers.
I'd never heard this, and it's odd if it's true because the numbers are always off when released. Media create numbers have already been updated but you can find them below. I don't know of an official site for Famitsu, but you can find the numbers posted about and linked in any number of places.

http://www.m-create.com/ranking/

Maybe the Famitsu is like an early estimate and the m-create ones are the final numbers if that's the case.


Quote:
VGChartz rips those numbers off. They were directly stealing them for a while, then NPD threatened to sue them so they changed to doing weekly updates and no longer claim to use NPD numbers, even though they still DO use them.

I read a post not that long ago on their site where they categorically denied that the NPD ever threatened to sue them, that all that ever happened was the were contacted by someone at the NPD inquiring as to how they got their numbers and that they responded with what they did and that was that.

Again this is a claim I hear a lot without any evidence. Can you provide a link? Show where the NPD said they had threatened legal action against VGChartz? Anything?



But really, at least you admit they probably do have access to actual sales data, which certainly gives them an advantage in terms of estimates over me. And they present the data in a very nice format. What does it really matter that it's not perfectly accurate? It's not like it's the end of the world if someone thought that one console won a month and it turned out the other did instead. And I always find the weekly sales numbers a breath of fresh air compared to most of the arguments that get repeated ad nauseam in these forums.
Reply With Quote
Reply

Related Topics
Thread Thread Starter Forum Replies Last Post
Defense Grid: The Awakening Master X Game Room 9 09-06-2009 02:58 PM
Final Fantasy Crystal Defenders ( AKA Tower Defense ) SupraDoom Game Room 5 08-07-2009 02:11 PM
In defense of Darkfalz! TheLastGenius Gaming Smackdown 5 07-07-2008 08:49 PM
Sony Defense Force MGS 4 review ! Leon Gaming Smackdown 9 06-04-2008 12:26 PM
Take-Two Directors Building Defense Against EA Gutter21 Gaming Smackdown 3 03-23-2008 06:58 PM


Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are Off
Refbacks are Off