Dell Inspiron with Overstressed Parts

I was given a Dell Inspiron 3451 that wouldn’t turn on.  Some probing around showed that power is getting onto the motherboard, but none of the switchers that power the CPU would turn on.  After some time doing this I noticed that board was getting warm near the HDMI connector, so I then began to focus on closely checking the parts in that area.

What I found was one of the switching power supply controllers had a hole in the package.  This can happen when a part overheats, or has to deal with an electrical overstress event (which also turns into overheating and package cracking/venting).

TPS51225 with a tiny hole in the package.
Circuit with broken IC.

Some more searching found a second suspect part, in this case what appears to be a diode with a pit in the package.

Package with pitting.

From what I can tell, this traces a path back toward the HDMI port itself.

Location of suspect parts.

I haven’t tested all the diodes in that path, but I’m guessing there was an electrical overstress event (ESD or surge) through the HDMI port, which decided to take a path through these parts to GND.

Is this an issue with the design? It’s hard to say for sure, but ideally the energy wouldn’t make its way into major components.  Usually there are protection diodes (possibly all the other parts in the top left) that should provide a fast path to the board’s ground plane, but in this case it didn’t turn out well.

Given the damage done, it’s probably not worth attempting to repair, since I don’t know what other parts might also be damaged and this is a low-cost laptop to begin with.  Best option in this case is to salvage parts to repair other things.

The Dell Venue that Destroys Batteries

I have a Dell Venue tablet which has a removable battery.  It’s a good thing the battery is removable, because I’ve now how to replace it once a year over the last two years.

Dell battery label.

Each battery is made of two LiPO cells.  Over time the battery cells begin to puff up.  Since there’s no room for the expanding cells, the battery begins to push on the back of the LCD, causing discolored blotches on the screen.  Removing the battery makes this issue go away, but why are the batteries dying so fast?

Bulging battery.
More bulging battery.

This battery was disassembled, and the monitoring PCB was examined.  I didn’t find anything unusual there.  The on-board circuit more than likely only protects the battery from being discharged too much, and from overheating (given the thermistor attached).

Dell battery monitor circuit.

The voltage on each cell was probed and measured 4.3V.  A LiPo cell is fully charged at 4.2V, showing that the cells are being overcharged.  Overcharging a cell will stress it and can cause the cell to puff up like mine have.

Unfortunately the problem is with the tablet and not necessarily with the batteries or protection circuit, and that means battery 3 will eventually meet the same fate.  I don’t have a solution for this yet except for attempting to contact Dell.  Since it’s out of warranty they probably won’t care, but it’s unfortunate they’ve released a product that prematurely destroys batteries.

“Paris” Prototype Apple Macintosh II Motherboard

I recently acquired this motherboard, primarily because it had NuBus connectors so I can explain to people outside of Apple fanatics circle don’t know what it is these days.  To my surprise, I discovered it is much more special and rare than your average Apple product.

A photo of the top:

Apple Paris Motherboard – Top

And a photo of the bottom:

Apple Paris Motheboard – Bottom

This has a few unique features:

  • It’s reddish-brown, which I’ve never seen in a released Apple product.
  • It doesn’t have much to denote it is an Apple product, other than a few parts on the board with the Apple name.   The board itself doesn’t say “Apple” anywhere.  Looking up the part number online yields nothing.
  • It looks similar to other Macintosh II boards, except in some areas parts have significantly moved around.
  • The copyright (1986) is a year before the release of the Macintosh II.
  • There are a couple debug wires on it.
  • There’s an Apple Computer add-in card in what would normally be a single IC socket.
PARIS MAIN LOGIC
Apple prototype board

Not being a true Apple fanatic, it took some digging to figure out that this is in fact a “Paris” prototype motherboard for the Macintosh II computer.  In fact, I have yet to find another “Paris” photo online anywhere!

Unfortunately it is missing most of the socketed parts, which were probably taken to build a “released” Macintosh when the final board was available.  Also the ROM chips are missing, but if my theory is correct the ROM was close enough to release that it wasn’t anything that unique.

What will I do with it now?  First I plan on taking some higher-res photos with a camera better than what’s in my phone.  Then I might take the time to try to build it up with parts I can find online.  Another option would be to sell it to a true Apple collector, but for now it makes an interesting conversation piece.

Converting a FreeNAS stripe to mirrored drive

RAIDZ is a powerful tool, and FreeNAS makes it easier to use via the GUI, but there are a number of advanced things you can’t do in the GUI alone.

In this case, I had an expanded RAIDZ-1 volume with a single striped disk (by accident), and I wanted to convert that striped disk to a mirrored one.  You could also use the same steps to convert a single striped disk (yes a single disk is still labeled as striped) to a mirrored disk.

There are a few other sites that provide the commands, but they don’t do a good job of explaining what the commands are doing.  I’ll try to clear that up!

Determine that name of the new drive.  You can do this within the GUI.  Disks are usually named /dev/ada0, /dev/ada1, etc. In my case (using a PERC H310 with Avago firmware) they show up as /dev/da0, /dev/da1, etc.  The newest drive will have the highest number.  You can also use the SMART test to verify the drive by serial number by running:
smartctl -a /dev/<yournewdrive> | more

Once you determine the correct new drive, remove any existing partitions with:
gpart destroy -F /dev/<yournewdrive>

When all partitions are deleted you can then create the ZVOL partitions.  First start by creating the partition table of type gpt with:
gpart create -s gpt /dev/<yournewdrive>

With the partition table created, you can now add the partitions. All drives get 2GB reserved for swap, and the rest can be used for the ZVOL.  First create the 2GB swap partition (partition 1), which starts at an offset of 128 sectors (which is reserved for the partition table):
gpart add -i 1 -b 128 -t freebsd-swap -s 2g /dev/<yournewdrive>

Now you can create the ZVOL partition (partition 2), which is essentially the rest of the disk space:
gpart add -i 2 -t freebsd-zfs /dev/<yournewdrive>

Next check your work:
gpart show /dev/<yournewdrive>

If you did everything correctly, you should end up with a table like this (in my case for /dev/da5, which is a 4TB drive):

[root@nas ~]# gpart show /dev/da5                                               
=>        34  7814037101  da5  GPT  (3.6T)                                      
          34          94       - free -  (47K)                                  
         128     4194304    1  freebsd-swap  (2.0G)                             
     4194432  7809842696    2  freebsd-zfs  (3.6T)                              
  7814037128           7       - free -  (3.5K)    

If you find you’ve done something wrong, you can always destroy the partition table and start over again.

With the drive partitioned, you can now add it to the pool.  This part is tricky, because you need to do it by GPTID, which is a long hexidecimal code.  If possible, do this using an actual SSH session to your server, because you can’t copy and paste (to my knowledge) to the shell available in the web browser.  To determine the GPTID of each disk, run:
zpool status

You will see something like this:

  pool: VOL1                                                                                                                        
 state: ONLINE                                                                                                                      
  scan: scrub repaired 0 in 0h17m with 0 errors on Sun Mar 19 00:17:55 2017                                                         
config:                                                                                                                             
                                                                                                                                    
        NAME                                          STATE     READ WRITE CKSUM                                                    
        VOL1                                          ONLINE       0     0     0                                                    
          gptid/c3b13e97-ef38-11e6-9be5-74867ad1a828  ONLINE       0     0     0                                                    
        cache                                                                                                                       
          gptid/f36a38d3-1744-11e7-8856-0090fa79871a  ONLINE       0     0     0                                                    
                                                                                                                                    
errors: No known data errors 

Those gptid/<GPTID> values are the descriptors you’ll need.  Find the one for the existing drive and copy it to a text editor (notepad).  Then run:
glabel status

Here you will see a list of every drive in the system with the gptid/<GPTID> names for each.  Find the one that matches the new disk (it’s listed multiple times for each disk, so make sure you select the GPTID that matches the ZVOL, not the swap partition) and copy that to notepad as well.

Now you will need to add the disk to the zpool:
zpool attach <volumename> /dev/gptid/<existing disk GPTID> /dev/gptid/<new disk GPTID>
where volumename = the name or your zpool volume.

If all went well, the disk will now be added as a mirror, and the system will begin to resilver (copy all data over to create the mirror).  You can check this in a number of places, but one of the easiest is the GUI.  Go to the storage pane, click on the volume, then click on the Volume Status button at the bottom.  You will then see status like this:

Resilver of an array after the mirror was added.

You can also run zpool status again, which will now show the disk in the list and indicate a resilvering status until complete.  The status light will also go critical in the GUI until the resilver is complete, but there is no reason to worry and all your data will be available during this process.

That’s it!  You’ve successfully created a mirrored disk array without having to wipe the original disk and start from scratch.

A summary of commands:

Get drive name: smartctl -a /dev/<yournewdrive> | more
Clear partitions: gpart destroy -F /dev/<yournewdrive>
Create partition table: gpart create -s gpt /dev/<yournewdrive>
Create swap partition: gpart add -i 1 -b 128 -t freebsd-swap -s 2g /dev/<yournewdrive>
Create ZVOL partition: gpart add -i 2 -t freebsd-zfs /dev/<yournewdrive>
Check your work: gpart show /dev/<yournewdrive>
Get GPTID of existing drive: zpool status
Get GPTID of new drive: glabel status
Add disk mirror: zpool attach <volumename> /dev/gptid/<existing disk GPTID> /dev/gptid/<new disk GPTID>
Check for resilver: zpool status

The Dead OBi200

After some thunderstorms came through, my OBi200 VoIP adapter stopped working.  The network end worked fine, but there was no longer any dial tone, and the device status page for the PHONE port no longer showed any information.

I only had the device for 11 months, so I promptly contacted support.  Since it was probably broken due to a surge/overstress event on the phone line, I decided to open it up and take a look.  I wanted to see if there was anything obviously broken that I could just replace and bring it back online, as well as I was curious what parts they had used in the design (and if there actually was any protection on the ports).

OBi200 Board

As expected, there really isn’t much inside.  There are three primary ICs:

  • Marvell MCU which provides the Ethernet interface, system control, config pages, etc.
  • RAM for the Marvell MCU
  • A Silicon Labs Si32260-FM1 ProSLIC telephone interface IC

The rest of the board is power supplies and a few components required by the primary ICs.  For what it’s worth there does appear to be an ESD protection IC on the USB port, but that’s good general practice for USB anyways.

I couldn’t find a full datasheet online for the Si32260-FM1.  The best I found was a couple-page datashort with a block diagram, pinout, and a summary of what the device does.  It essentially is everything necessary to provide a VoIP interface, including phone line voltage generation, DSP to encode and decode analog and FAX data, and a simple SPI interface for digital data transport (in this case to/from the MCU for transfer over Ethernet).

The SI3226x block diagram

Note that the block diagram shows two channels, but the OBi200 only provides one phone channel.  It appears that the second channel is connected and populated, so getting a second phone port is probably just a firmware change.

Unfortunately the datashort available didn’t have a reference circuit vendors usually provide, which more than likely is what Obihai used in this design.  From what I can tell (without a bunch of probing to determine actually connections) there appears to be a simple analog filter on the frontend made of 0805- and 0603-sized components.  The resistors appear to be either thick- or thin-film and the caps all MLCC.  A quick check with my DMM didn’t find anything that was obviously open, short, or different than a neighboring part with a matched circuit shape.

I did not see anything in the way of TVS diodes, spark gaps, or any other component that would provide significant protection from a high-voltage transient event, which is somewhat unfortunate.  Part of this is probably due to the small size, and the other due to there not being an actual ground lug anywhere on the product (the GND of the power port through a wall wart isn’t a true GND).

If I were to redesign this, knowing it’s probably going to connect to a set of phone lines that might be connected to a network of phone cable where lightning could possibly couple in, I would have probably added at least a couple TVS diodes and a GND lug.  Most customers probably wouldn’t connect the GND lug, but it’s better than nothing.

Fortunately, there are surge suppressors for phone lines available,  but of course it’s a separate product that needs to be purchased.  I decided to go with a Tripp-Lite DTEL2 suppressor, which connects between the OBi200 and the phone network in my home.

Tripp Lite DTEL2 Surge Suppressor

In the end, Obihai honored their 12-month warranty, and I sent the broken device back before doing any additional debugging.  I can only hope that adding an external suppressor will avoid another failure in the future.

Setting up CrashPlan using VMWare and FreeNAS

I’ve had CrashPlan on my list of apps to get installed in order to better backup my system data that isn’t already installed on a NAS.  The FreeNAS forums allude to difficulty involved in properly setting up and upgrading CrashPlan as it isn’t extremely straightforward, plus getting clients to connect to a FreeNAS-based CrashPlan instance also seemed hacky (mentions of changing config files, etc).

I gave the server install a try and dealt with a number of missing driver dependencies, issues with bash properly running, and a bunch of other annoying things.  Then I decided to try a different way.

Crashplan has clients for Windows, Linux and Mac.  In addition to FreeNAS, I also have a VMWare server that is already running an instance of Windows that is providing a few services that weren’t available in other OSes.  I decided this might be the easiest way to go.

One downside to this is I didn’t want to store the backup data on the VMWare server.  The volumes on that server aren’t as large, and while it is setup with RAID volumes, they don’t currently have the 2x redundant backup that my volumes on FreeNAS currently take advantage of.

The easiest solution would be to mount a network drive on the FreeNAS server, but for some reason CrashPlan doesn’t allow a network-mapped drive to be a backup destination.

Fortunately we are running this in VMWare, and there’s more than one way to mount a drive.  What I ended up doing was creating another VMWare disk volume that is stored via NFS on the FreeNAS server.  This is then mounted as a “local” hard drive within the Windows VMWare instance.  The disk was created with 250GB space and thin provisioning, which should be more than enough for my needs.  And if I ever run out of space it’s easy to grow drives.

After this I was worried that the disk image might be replicated as a full copy in FreeNAS each time it detects that the file has changed, but it appears that snapshots and replication are able to keep track of differences and only use space required to replicate the changes.

So in summary, to avoid issues with running CrashPlan in a FreeNAS jail:

  • Run a Windows installation in a VM (I used VMWare)
  • Create a second virtual disk using the network storage as the disk location (if not on the same machine)
  • Mount this disk, partition, and format it like any other disk in Windows.
  • In the CrashPlan app within the Windows VM, set the new disk as the default location for CrashPlan backups
  • Enjoy!

How NOT to setup a RAIDZ Volume

I’m relatively new to FreeNAS, which I somehow missed out on for many years.  Now that I found it I’ve moved over to using it as much as possible for my storage needs.

Based on a number articles it seems as though it’s easy to add storage any time you like.  This is true, BUT there are some HUGE caveats to this.  The FreeNAS GUI makes it somewhat hard to make a mistake, but if you think you know what you’re doing you can override these features and actually put your data at risk.

I started out with three drives and created a RAIDZ-1 volume.  For those unaware, this means that if any ONE drive fails then all the data will still be safe.  To be even more safe, it’s recommended to use a RAIDZ-2 configuration and/or backup your data to yet another location.  I decided on the latter, and have backed up my data to a second FreeNAS server (via snapshot replication).

So far I’ve followed generally-accepted good data integrity practices here, but now for my mistake.  I decided to buy another drive of the same size, and I wanted to add that to my server to increase the space available.  The FreeNAS GUI wouldn’t let me add it with the basic Volume Manager utility, so I went to advanced mode and added it to the ZVOL.  What I ended up with was this:

Adding an additional drive to VOL0 resulted in a striped disk configuration, which is bad news here.

The three original disks (da0-da2) maintain their RAIDZ-1 state, but now a portion of the data is shared with the striped disk da3.  The striped disk has NO redundancy.

Note that the cache drive ada0 is a SSD providing a L2ARC to the volume.  Cache is usually a single striped drive, and if the data is lost there then no harm will be done to the system.

What wasn’t explained very well (or I failed to read earlier), is that RAIDZ volumes can be added to, but each group of disks, once configured, remain in the configuration that they were initially created with.  What I had done was EXPANDED the storage available, but STRIPED my original raidz1-0 array with a single disk.  Because the expanded storage is a SINGLE disk, if that one disk fails then the entire volume will be destroyed.

So if any single one of da0-da2 fails then the array continues to operate in a degraded state. But if da3 were to fail, then the entire array goes down!  Kind of kills the point of RAID, right?

The reason for this is that RAIDZ is a high-performance datacenter-quality product.  Datacenters aren’t usually adding single drives, but rather entire arrays of disks when they need to increase storage.  If you decide to use FreeNAS and RAIDZ, then you must keep this in mind.  It does take more planning (and money for additional disks), but that’s the tradeoff of using this tool.

So how do I solve it?  Well right now da3 needs some redundancy.  I ordered another drive of the same size, which will then be a mirror for this disk.  Note that the mirror must be added via the console, not through the GUI.  There is no GUI-based way to convert a striped disk to a mirrored disk, even though the FreeBSD tools support it.

So then I will have a RAIDZ-1 array + expanded storage of a mirrored array, all sharing data of VOL0.  This still has some limitations on redundancy though (assuming disk 5 is named da4):

  • If any one of da0-2 fails, then the array is OK.
  • If any one of da3-4 fails, then the array is OK.
  • If da3 and da4 fail then the array is dead (the mirrored array has completely failed).
  • If any two of da0-2 fails, then the array is dead (the raid-z1 array has completely failed).

The likelihood of this happening is low, plus all my data is still backed up on a second server.

Another option (since all my data is backed up) would be to destroy the array, and rebuild it with all 5 disks (da0-3 + the one that will arrive soon) in a RAIDZ-2 configuration.  That way if up to TWO of ANY of the disks fails then the array would continue to operate.

I haven’t yet decided if I will do this, but for now adding the mirrored disk fixes my mistake.  Don’t do what I did.