Monday, June 11, 2012

Lenovo sales/support can't do their jobs if Lenovo don't tell them the facts

I just bought a Lenovo T420. Of course, the T430 promptly came out, but that's not why I'm very angry at Lenovo. This is:


1802: Unauthorized Network Card is plugged in - Power off  and remove the miniPCI network card (413C/8138).

System is halted


UPDATE 2012-11-17: It's been six months, and I'm out of patience. I'm filing an ACC complaint..


UPDATE 2012-06-24: I've received the replacement 3G card from Lenovo, and it works well. The Gobi 3000 based Sierra MC8790 card is a bit ... challenging ... under Fedora 17 and I wish I'd just been able to use my existing, working card. Still, this one should be lots faster when it's working, and it works under Windows at least.

They still haven't updated their website, so it's going to be nagging-time soon. Not impressed.


UPDATE 2012-05-15: After some help by lead_org on the Lenovo forums I was able to get in direct contact with Lenovo ANZ's customer care manager, customer care team lead, and social media managers. An amazing three hours after my initial email, I received a helpful reply. I must paraphrase as I don't have the writer's explicit permission to quote the full message; it indicated that:

  • This was the first time the Australia/New Zealand group had seen this issue
  • They were sorry for the disruption and would courier me a compatible 3G card
  • The website would be fixed to inform potential customers of the restriction, and would educate their sales/support staff about it
  • (I'll quote this bit; it's a formula match for the line I see everywhere else on this topic):
    With regards to the whitelist, it has existed since the introduction of modular cards - Lenovo have to qualify particular wireless devices not just internally but with the FCC and other telecommunications bodies around the world, hence the control around this list exists. As such there is no way to bypass this list as it would effectively allow violation of legal statutes with regards to telecommuncations devices.

Kudos for a quick response, mea culpa, action to stop it happening in future and a gesture to make it right for me. Good on them.

I'm less thrilled by the vague and waffly detail-free explanation of the whitelist; I've been unable to find anywhere where Lenovo or a Lenovo rep has clearly stated which regulations affect them, but not Dell, Acer, or numerous small vendors. I asked for clarification on this point but have received no reply to date.

The important thing is that they'll fix their website and rep training to make sure others aren't mislead; in the end, their reason for the whitelist can be "because we can and because we feel like it" and so long as they're up front about it that's less bad.

Related writings by others:


Note the common theme: complete surprise that their laptop refuses to work, because they had no warning about the lockdown until they already had hardware for it. Not cool.


That error informs me that Lenovo has chosen not to permit me to use my 3G cellular modem card in my laptop. I'm not leasing this thing off them, I bought it outright. This isn't a technical limitation or restriction, it's a policy decision enforced by the system's EFI firmware or BIOS.

Before I bought this laptop, I'd heard that Lenovo and IBM before them used to restrict installed Mini-PCI-E and Mini-PCI cards in BIOS, refusing to boot if a non-Lenovo-branded 3G or WiFi card was installed. I had a 3G card I wanted to use, and anyway didn't want to be part of that sort of thing, so I called to confirm they didn't still do that.

The sales rep assured me in no uncertain terms that there is no such restriction. I was promised that a Lenovo T420 will boot up fine with a 3rd party 3G or WiFi card installed, though of course they can't guarantee the card will actually function, I'd need drivers, and they won't provide tech support for it. Fine.

I didn't completely trust the sales person - though they knew what MiniPCI-E was, so they were ahead of the curve - and called support. I asked them the same thing: Would the laptop refuse to boot and give me an error in BIOS if I put a non-Lenovo 3G or WiFi card in. I specifically asked about whether it'd give me a BIOS error and refuse to boot. Not only did they assure me it wouldn't, but they said the card would work out of the box if it were the same model as one of the ones Lenovo sells. No misunderstandings here, we weren't talking about USB stick 3G, but mini-pci-E.

At this point, let me propose to you a game. Be nice, though, it isn't the fault of the sales and support reps, they're getting mislead and misinformed to as well. Go to shopap.lenovo.com or your local Lenovo online store, or call their sales number for your region. Ask the poor innocent who is just trying to help you whether you can use a 3G card from your old Dell laptop in the T420 you're thinking of buying, since it's a mini-pci-E card and you have all the drivers. Help them; mention that you've heard that Lenovo and IBM before them used to stop machines starting up if there was a non-Lenovo wireless card in them. Ask them if that's still the case. Copy and paste your chat to the comments if you feel like it, but be very sure to blank out the rep's name or I'll get really angry at you too.

Betcha they'll tell you everything will be fine. Mine did and I have a record of it.

It seems Lenovo's corporate decision makers don't tell its sales and support reps everything they need to know:



That's from my new Lenovo T420, a great machine except for the whole locked-down-so-you-can-only-use-our-branded-hardware thing. (OK, and the lack of USB3/bluetooth4, but we can't have everything).
Of course, it's not surprising the sales and support folks don't know since the issue isn't documented anywhere in the tech specs, so-called datasheet (sales brochure) for the current model T430, or the T420/520(US) or T420/520(AU) etc. here they advise you to select a "wireless upgradable model" but don't bother to mention the card'd better be from them, Or Else.

It isn't even in the hardware maintenance manual! Points for Lenovo for publishing this, unlike most vendors, but it clearly needs to be a wee bit more complete.

There's even a knowledge base article about it, but you have to know the error code to search for before you can find it. The ThinkWiki article on it wasn't obvious either, isn't on Lenovo's site, and isn't something you're likely to find until you've already been burned.

The only official mention I found outside that knowledge base article was a tiny footnote on the product guide for resellers, where it says on page 160 (not kidding) regarding 802.11 WiFi:
Based on IEEE 802.11a and 802.11b. A
customer can not upgrade to a different
wireless Mini PCI adapter due to FCC
regulations. Security screws
prevent removal of this adapter. This wireless
LAN product has been designed
to permit legal operation world-wide in
regions which it is approved. This product
has been tested and certifi ed to be
interoperable by the Wireless Ethernet
Compatibility Alliance and is authorized
to carry the Wi-Fi logo.
and on 161 regarding "Wireless upgradable" 3G/4G (terms "3G" and "4G" not actually used in document):
Wireless upgradable allows the system
to be wireless-enabled with an optional
wireless Mini PCI or Mini PCI Express
card. Designed to operate only with
Lenovo options.
None of this handy information was, of course, on the store page, laptop brochure, or specs where customers might actually see it, and was unknown to their own sales and support reps, who've been kept completely in the dark. Nor does it say, even in the above sales-rep-info footnote, "firmware will disable the laptop if 3rd party card detected"; it says it's only designed to work with Lenovo parts, not that it'll actively and aggressively stop you you using non-Lenovo ones.

Not happy. Their sales chat people on the website assuring me that the sky is purple and full of ponies is one thing, but their support folks not knowing this is something else entirely. It's not their support folks' fault their employer is lying to them, but it most certainly is the fault of Lenovo management for enforcing this policy and keeping it secret from the support and sales teams.

What makes me angriest is that the website for 3G capable models doesn't say anything about this restriction, nor is there any link about it regarding WiFi cards, or any mention in the specs for the MiniPCI-E slots or antennae, or ANYWHERE.



"Datasheet" excerpt specs from the T420. No sneaky little asterisks. No small text elsewhere.




I've been in touch with Lenovo and asked them to make this right. To their credit, the support rep I spoke to understood that this isn't cool - and was very surprised about the issue. They immediately took detailed notes, got all my details, and escalated to customer care. The customer care person didn't really get it, but did take it seriously, and said they'd look into getting me a replacement compatible card at Lenovo's cost (lucky I don't need this specific one for app compat!) and getting the website fixed.
Whether it happens, well, we'll see.

Meanwhile, Lenovo continue to justify this practice when it comes up with the claim that "every manufacturer does it" even though neither Dell nor Acer seem to; unsure about others.
I've explained that their options to make it right are in two parts: make it right for me, and make sure it doesn't happen again.

For making it right with me, the options I see are:
  • Replace my card with a Lenovo-branded equivalent that's compatible
  • Issue a firmware update that removes the restriction
  • Refund me and let me return the laptop so I can buy from someone without the restriction. (I don't actually want to take this option because all laptop makers are equally evil, and in all other ways the T420 is great.).
  • Replace my T420 with an equvalent T430 with Lenovo brand 3G. Yup, that'd probably calm me down plenty.
For the second part, they need to:
  • Issue public firmware updates removing this restriction from the ThinkPad line and link to them from the knowledge base article; or
  • Add a link beside all "3G ready" or "mobile broadband ready" text in the shop and sales documentation explaining that only Lenovo brand cards are permitted. Update the laptop spec page for the mini-pci-E and wifi to note the restriction there. Agree to swap cards or issue custom firmwares for people who're bitten by this who bought before the shop was fixed.
I give myself about 90% on getting it fixed for me because I'm quite happy to go to small claims about this and I have written records. As for the second part: I'm persistent, and I'm very, very angry.

PostScript: Here's the chat log I got yesterday when - as an experiment - I tried an online sale person to see if I'd just been unlucky when I was assured by the (much more competent but harder to copy and paste) phone sales and support people that it isn't a problem. It's pasted from the chat window. I have screenshots too. I've masked the rep name from the log for obvious reasons.


Connected to xxxxxxxxx         xxxxxxxxxx

Thank you for contacting Lenovo Sales Chat. My name is xxxxxxxxxxxx. Happy to help you today.


Connected to xxxxxxx xxxxxxxxxxx
Hi. Sorry to trouble you, but I'm about to order a T420 I've saved in my cart and I have one final thing to clear up before I order it.
Hello, I will be more than glad to assist you
I've been told that Lenovo laptops (and IBM laptops before them) used to refuse to start up with a BIOS error if you put a 3G or wireless card in them that wasn't made by Lenovo.
Does that still happen?
I have a 3G card from my Dell I want to use in my new laptop. I can take care of the drivers, etc, all I need to make sure of is that the Lenovo won't actively refuse to start up with the 3G card installed.
I am unaware of such issue, we have not come across this issue till now
You can go ahead and place the order.
That issue could be a customer review, that must not be a problem with the machines these days.
OK, so if I put my 3G card into the T420 it will start up? (I know you can't guarantee the 3G will work, I just need to know the laptop will start)
Excellent, I'll order.
Yes, it will work

NOTE: Rep then tries to convince me to go ahead and buy now; I evade. I feel a little bad about this, but it's only taking a moment of their time to do this chat and it'll help stop people getting stuffed over later. I thought I'd give them another chance with a GIANT hint, so I posted the support article about the BIOS error. At least they didn't try to brush that off too:

Oh! I just found this:
It says "All Thinkpads" are affected. That's not good.
I have just checked it, well in this case I will provide you the contact number of Lenovo Technical support
As the information is updated 1 year ago.
You can check for the current one with Tech support and place the order
Sorry for the inconvenience
131 426 is the contact number of Lenovo Technical support and they work Monday through Sun 9 am - 9 pm Australian EST
No worries, I'd rather be sure. I'll call them.
Thank you

2 comments:

  1. This sort of thing is a real and general problem in our chips-in-everything replace-not-repair world where the encryptors have a decisive advantege over the decryptors. I weakened and bought an iPhone a few years back; keeping that thing even relatively "open" is a nightmare. We have open-source software but open-source hardware is a different matter. Raspberry Pi is an encouraging but tiny step.

    ReplyDelete
    Replies
    1. ... except that Rasperry Pi is based on a locked down Broadcom platform where the closed-firmware GPU with closed (now open but very limited) driver is in control, and the CPU only gets to do what the GPU says.

      I don't find it particularly exciting compared to mini-ITX or other more open solutions.

      Delete