OpenSolaris Live Hadoop (OSLH) Release Notes Version: 2008.08.28 What's New in version 2008.08.28 -------------------------------- This is the initial release of the OpenSolaris Live Hadoop, and includes: - OpenSolaris snv79, including sparse Zone support - Hadoop 0.17.1 - Java build 1.6.0_03-b05 System Requirements ------------------- - an x86 or x86_64 based PC or Mac computer - A CD-ROM or DVD drive - At least 1 GB of memory - To install on the local disk, you should have 10 GB of free space Booting/Installing/Uninstalling ------------------------------- Place the CD into your CD/DVD drive, then either turn on or reboot your computer. Select the CD as the primary boot device. This procedure varies from vendor to vendor, however on Apple computers you can press and hold the 'c' key during bootup. Known Issues ------------ 1. Ensure that you have sufficient available memory. We have tested OSLH with memory configurations ranging from 1 to 2 GB. 1 GB is the basic amount of memory required to bring up the three-node Hadoop cluster and run the 'wordcount' demo on a small amount of data. Submitting more complex Map/Reduce tasks and/or operating on more data will require more memory. One behavior you may notice in low-memory configurations is that Hadoop datanodes may take an unusually long time to join the Hadoop cluster. See issue 2, below, for more information on this symptom 2. Hadoop starts up, but the cluster does not contain any datanodes One behavior you may notice in low-memory configurations is that Hadoop datanodes may take an unusually long time to join the Hadoop cluster, and so the cluster appears empty. The remedy we have used is to allow more time for the cluster to come up (up to several minutes). Make sure that before you attempt to read or write data to HDFS, that you ensure that the cluster is operational via: bin/hadoop dfsadmin -report This command should return a list of two active datanodes. 3. Installation of the Live CD image to the local drive does not properly configure Hadoop This issue is due to a mismatch between the list of users on the Live CD and the users that are created when you install OpenSolaris on a local disk. To start up Hadoop after installation, you will need to: - Log into each zone and create a new user that has the same name as the user you created during installation. For example, if you created the user "Alan Turing" with userid "aturing" during installation, you will need to create the "aturing" user in each zone. - You can log into each zone by becoming root, and typing "zlogin node1" to log into node1, and "zlogin node2" to log into node2. - Next you will need to ensure that you can log into each zone without typing in a password. To do this, you must create an ssh key/pair that does not have a password, then propagate this password to each zone. - To create the password-less key-pair, type: ssh-keygen -t dsa -P '' -f ~/.ssh/id_dsa cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys - To propagate the key, 'scp' it to each zone: scp -r ~/.ssh node1: scp -r ~/.ssh node2: