晴朗星空下的網絡|The Network under the Bright-Bright Sky

I learned a hard lesson, and is now paying off.

I can’t believe what I typed before my eye, I just did “rm -fr public_html”. A Ctrl-C stopped bleeding, but it doesn’t help from losing 1GB+ of files.

RAID won’t help in this case, but I have backup.

假設硬盤的生長速度緊貼Moore’s Law (so far還可以)，而我的資料的生長速度比Moore’s Law慢，那我的server在未來一定可以還可以裝得下我我的資料。

If the harddrive growing speed is strictly sticking with the Moore’s Law (so far that’s pretty much the case), and that my data is growing slower than what’s Moore’s Law has predicted, then my server could still hold my data at anytime in the future.

但大前提是我要一直買新的Harddisk。

A big assumption: I have to keep forking out money in purchasing new harddrive.

2007年初拆巨資買下的500GB已經爆滿了一段時間，我也不得不把我的資料增長暫緩了好一段時間。眼見1TB雖從初期的$3000回落到$13xx，但感覺還是有點貴，而且我的資料增長束度好像有點放緩（少看了動畫的緣故吧？），所以前天買了一個640GB的先充充饑。

In the Q1 of 2007, I spent a whole lot in buying a 500GB, and it was full for quite some time. I have to stop my data from growing for a while also. While the 1TB harddrive price dropped to HKD $1300 from HKD $3000 when it was released, it’s still pretty expensive. Besides, it looks like my data doesn’t grow as fast (because I watched less Anime recently? who knows-), so I bought a new one the day before and settled with a 640GB.

舊的500GB就轉移過去Backup Server，Backup Server現在就有2*250GB+1*500GB=1TB的空間。比我19年前用在386機上的100MB大了整整10000倍（算起來每17.1個月就翻了一翻，雖不是蘋果比蘋果但anyway）。Backup上的320GB還可以退下來給我Backup Laptop用的說。

The old 500GB is then transferred to the backup server. The backup server now have 2*250GB+1*500GB=1TB space available. Comparing with my 100MB  installed in a 386 desktop 19 years ago, it is 10000 times bigger. (That’s about doubling for every 17.1 months, that’s not apple to apple comparison, but anyway). The 320GB from the backup is also offlined so I colud use it to backup my desktop now.

花了整整1天，才能把4xxGB的data 通過100Mbps Ethernet backup一次，看在下次更新時要把GE network card換上去backup server才成。

It took 1 day to re-sync the existing 4xxGB through the 100Mbps ethernet. Looks like I must upgrade it to GE next time.

若然未報，時辰未到……

大概是我閒時沒多感恩，或許初一十五沒有燒什麼Code或紙札Developer孝敬電腦神，今天就捉弄了我一下以示警戒。

本來正想Upgrade去Wordpress 2.6，主要還是因為要測試一下xLanguage的相容性，當然第一部就係將最新的資料dump過去Testbed平台上。很久沒有做這個動作，幸好之前做轉移的時候都留下一個script，跑一下，OK沒問題。

之後把我修訂過的部份Diff出來，再把Wordpress 2.6的文檔按指示覆蓋上去，再把之前的Diff patch上去，都沒有問題。

Login進去，看看我之前修訂過的部份是否真的能成功Patch上去。但還沒有查Patch的問題，就發現好像有些條目不翼而飛。回頭看一看轉移用的Script……

「大檸樂……」一陣涼風吹過我的一把冷汗，感覺份外清涼……似乎我把方向弄反，把老舊的Testbed轉移到Production Blog Database上。

還記得我的Daily backup server好像Down了一陣子，本來打算捱到回港才修理……循例試一下，希望有奇蹟出現，電腦卻冷冷的回傳一句「ssh: connect to host backup port 22: Connection refused」

查一下MySQL的Log Folder，完來電腦神還沒有完全離棄我。我之前Turn On的Binlog還真的是有效，一句一句的MySQL Statements還是可以讀出來。

稍為花了點時間找對了上次轉移Blog的時間，重播一次之間所有的MySQL記錄。要重播兩個月的SQL Statements還真的要花點時間，把無關痛癢的WassUp log table刪走後還是要一個小時左右才完成這次Replay。

今次教訓再次提醒Backup的重要性，按非正式統計，90%的資料災難都是人為引致的。還是要感謝電腦神留我一線生機，另外就是要快點修復Backup Server才成。

…it’s just a matter of time when you will be punished.

It’s probably because I didn’t have a gratitude, or that I didn’t offer sacrifices to the god of computer on full-moon and new-moon, and I got punished today as a warning.

I was upgrading the Wordpress to 2.6, primarily for testing the compatibility of xLanguage. Of course, the first step is to dump the existing data to the testbed platform. It’s was a very long time when I was doing this last time, but luckily I have left the migration script somewhere. Bang! It’s still working, great!

Then I “diff”ed the part that I customized, and overwrote the existing binaries with 2.6’s as instructed, then patched the “diff”, all green.

I was going back to the Admin panel, was trying to see if the patch still works on 2.6. Before I could get to there, I found that some entries are gone! Now when I was checking the migration script…

“Darn…” I felt an icy wind was breezing over my cold sweating face…Looks like I have managed to reversed the migration direction, I had the production blog database overwritten with an very old testbed data.

I still recalled my Daily backup server was down for a while, original plan is to leave it there until I went back home. And just now, I was pinging it while hoping miracle to happen. No it didn’t happen, the computer returned “ssh: connect to host backup port 22: Connection refused.”

Let’s take a look at the MySQL’s log folder, looks like the God of Computer didn’t abandon me yet.  The Binlog that I turned on earlier is still functioning; I can still dump every SQL statements ever executed on the server.

It took me some time to locate the last blog migration from the binlog, then replaying all the MySQL statements in between. Well, it actually took much longer than I expected to replay 2 month worth of statements, even though I had filtered out the WassUp unless transaction, it was still taking it more than 1 hour to replay the remaining statements.

Take home message: it’s important to have backup, and probably multiple backup. According to a non-scientific survey, almost 90% of data disaster was introduced by Human. Thank god for guiding me over the process, and I should fix my backup server ASAP.

數天前看見Newsgroup的文章提到某大討論區伺服器當掉了：

我們經歷了一個比上次災難還災難的事故，服務DCHome 4年的server底板終于告別我們，但衰收場，臨死之前比D不明信號我地既兩張 Raid 卡，同時隊冧兩組 Raid 5……其中SCSI Raid card 亦陪佢一起上埋路添。所以，我地幾乎成部server換過曬……

自己的Pocket PC，某天插上Cradle螢幕突然黑白間閃，拿起再插上去它就給我一個很華麗的歡迎畫面，換句話說：Hard Reset了，幾乎一切都回復至出廠狀態，包括所有資料和裝過的軟件。

幸好，之前設定了使用快閃來存放聯絡人、行事曆等資料，那裏是舫舟，可以避過Hard Reset的洗禮。嘿！只要到控制台把那個設定啟用一下，資料便跑回來。隨即放上Cradle，ActiveSync一下。很好，只要將Pocket PC的Computer Name改回慣用的那個就一切依舊。

看到ActiveSync有一個Full Backup的選項。呀，竟然預設是關閉不用的嗎？不要開玩笑，二話不說做一次完整備份。苦等了三分鐘後終於完成，繃緊的心情隨即得到舒緩。

反正有了備份，就試一下再Hard Reset看看有沒有問題，下次再出事時使好對付。Reset了，ActiveSync辨認到一個新裝置，選還原……五分鐘後進度棒終於跑到終點。完成時彈出一個對話盒，說有些檔案在使用中，不能夠覆寫，問我要繼續還是放棄，這種情況在Microsoft的產品也很想見，按了幾次重試也無效後就不以為然放棄了，反正那些檔名驟眼看都是dll檔。

自動Soft reset後，ActiveSync以慣用的Profile與Pocket PC連線，檔案總管下也看到了My Documents沒有異樣。到聯絡人看看……！？一個人也沒有，行事曆也是一個事件都看不見……在控制台弄一下快閃儲存的設定，能夠把行事曆的資料呼喚出來我很開心，但是聯絡人還是空空如也。即使完整備份回復的程序出問題，電腦ActiveSync也應該載有聯絡人過往的同步記錄吧？

再試一下回復，發現剛才的對話盒內列出的檔案中的，竟然出現Contacts Database、Appointment Database的檔名……即使更新了ActiveSync也沒有效。唉，怎會有這麼爛的回復工具，幾乎不可能的事總是要發生。[1,2]

冷靜想一下，Google一下。此刻才發現沒有Outlook或者對應的個人資料管理員的話，ActiveSync是不會將聯絡人等資料進行同步。另外就是回復覆寫的問題很多人都遇過，但就是沒有解決方法。第三，備份格式大概是一個封閉格式，坊間流存的兩個讀檔器 (STGViewer和MobileVault) 都試過，但就是不能夠成功將那些Database抽出來。其他人的解決辦法是在回復後，靠同步在Outlook中的資料來覆寫Pocket PC上的空檔。

最後還是乖乖的認命，接受摩菲定律的裁決。

結論，備份除了要常做外，還要確定回復是無問題。

[1] 感謝天地沒有在我猛再試回復->Reset的過程中把其餘在快閃的個人資料都破壞。
[2] 很久以前用Palm，即使Hard Reset了也只要再放上Cradle便輕鬆搞定。沒想過ActiveSync連這個基本中的基本也是這麼令人深感遺憾。若果有可行的選擇的話，下次要選一部跑Linux的……