Lab IT

From ZhangLabWiki
Revision as of 22:34, 26 June 2013 by >Dinh (→‎TSCC cluster)
Jump to navigation Jump to search

Register a new computer to the campus network

Web-Based Monitoring and Alarm Notification Web600

  • [1]
  • Port 4.1.34 B
  • admin:Zhang lab's common password
  • 132.239.25.73
  • gateway 132.239.25.1
  • netmask 255.255.255.0
  • nameserver 132.239.0.252

NAS

  • Apple Time Capsule 877b54
    • MAC address: 00:1F:00:1F:5B:87:7B:54
    • Wireless password: Zhang lab's common password.
    • Backup of lab wiki, also serves as a wireless router.

Wiki server

  cd /etc/httpd/
  sudo htpasswd passwords new_user_name
  
  Edit the httpd.conf file.
  Add the new_user_name to the following line:
  Require user kunzhang ...some_existing_user_name
      • Wiki backup
  Launchd deamon:  /Library/LaunchDeamons/labwiki.backup
  Script for backup: /Library/Scripts/wiki_backup.sh
  • Server migration from Mac Mini to Mac Pro: Migration Log
  • Reads from genome-miner are shared in /Volumes/genome-miner-SeqStore2/ (See [2])
  • MediaWiki was upgrade from v 1.13.2 to v 1.15.1
    apachectl stop
    #backup the mysql databases
    mv LabNotes  LabNotes.v1.13.2
    tar -xzvf mediawiki-1.15.1.tar.gz
    mv mediawiki-1.15.1 LabNotes
    cp LabNotes.v1.13.2/LocalSettings.php LabNotes
    cp -r LabNotes.v1.13.2/extensions LabNotes
    cp /LabNotes/AdminSettings.sample /LabNotes/AdminSettings.php
    #edit the /LabNotes/AdminSettings.php file for the admin previlege
    cd LabNotes/maintenance
    php update.php --aconf ../AdminSettings.php
    cd ../..
    mv LabNotes.v1.13.2/upload LabNotes
    mv LabNotes.v1.13.2/zhangloupload LabNotes
    apachectl start
  • Sphinx 0.9.9 full text search engine installed (Jan-23-2010). Updated on 10-Apr-2013 (/Users/kunzhang/www/LabNotes/extensions/SphinxSearch).
  • MediaWiki was upgrade from v 1.15.1 to v 1.18.2 (April-08-2012)
    mv LabNotes  LabNotes.v1.13.2
    tar -xzvf mediawiki-1.18.2.tar.gz
    mv mediawiki-1.18.1 LabNotes.v1.18.2
    cp LabNotes/LocalSettings.php LabNotes.v1.18.2
    cp -r LabNotes.v1.13.2/extensions LabNotes
    cd LabNotes.v1.18.2/maintenance
    php update.php
    sudo mv LabNotes/upload LabNotes.v1.18.2
    sudo mv LabNotes/zhangloupload LabNotes.v1.18.2
    sudo mv probedesign ../LabNotes.v1.18.2/
    sudo mv PeakPicker.jar ../LabNotes.v1.18.2/
    sudo mv SeqScannerInstall.exe ../LabNotes.v1.18.2/  
    sudo mv RobertsLabNotesSupplement ../LabNotes.v1.18.2/ 
    mv LabNotes  LabNotes.v1.15.1
    mv LabNotes.v1.18.2  LabNotes
    download the latest version of Sphinx extension: SphinxSearch-MW1.18-r92378.tar.gz 
    Extract the tarball and copy all the files to LabNote/extension/SphinxSearch folder.
    To rebuild the mysql database from mysql dump:
      /usr/local/mysql/bin/mysql -u wikiuser -pPASSWORD wikidb < wiki_db_current.sql
    Update on 2/6/2013
    I installed the latest version of mysql5 using MacPorts
      sudo /opt/local/bin/port install mysql5-server
    Note that the mysql files were installed at /opt/local/bin
    To start the MySQL server on launch, 
      sudo launchctl load -w /Library/LaunchDaemons/org.macports.mysql5.plist

Meangenemachine

  • Duel Intel Xeon X5645 (12 cores), 32 GB RAM
  • Booting from 500GB drive
  • Drive space: 600GB_store, 1TB_store1, 1TB_store2, 2TB_store1, 2TB_store2
  • Ubuntu 11.04
  • For general use
        • address: meangenemachine.dynamic.ucsd.edu
  • Setting up DHCP network:
    • Request DHCP-only networking with ACS
    • In: /etc/network/interfaces
auto lo eth0
iface eth0 inet dhcp
iface lo inet loopback
    • Restart networking
sudo /etc/init.d/networking restart
    • Get IP
sudo dhclient

Genemapster

  • Dual Intel Xeon X5520 (8 cores), 32 GB RAM, Tesla C1060 GPU
  • Ubuntu 10.04.3 LTS
  • For general use and for running GPU softwares
        • address: genemapster.dynamic.ucsd.edu

Genome-miner

  • Recently reinstalled using MacOS instead of Ubuntu; see notes here
  • A MacPro (8 core, 16GB RAM, 4TB), dedicated for GA Pipeline (PFBH406)
    • MAC addresses:
      • Port 1: 00:25:00:ee:6f:c8; connected to the campus network;
        • IP=132.239.135.41;
        • Subnet: 255.255.0.0
        • Gateway: 132.239.135.1
        • DNS: 132.239.0.252, 128.54.16.2
      • Port 2: 00:25:00:ee:3f:85; connected to GA PC;
        • IP=192.168.137.11
  • Installation of Ubuntu 9.04 (AMD64 Desktop).
    • Installed rEFIt for dual booting (http://refit.sourceforge.net/).
    • Installed Ubuntu 9.04 Desktop from ISO image.
      • Turned out that all the hard drives were not recognized with the RAID card. Removed the RAID card.
      • Press "c" to boot from the CD.
      • The OS was install in the hard drive in Bay 4.
      • The other two hard drives in Bay 2&3 were also reformatted to ext3
    • Ubuntu booted up correctly, but without network connection. Turned out that the Ethernet card was too new.
      • Downloaded the driver from sf.net/projects/e1000, manually changed the following two files, compiled and installed the driver.
   Added the following to the netdev.c: 
        {PCI_VDEVICE(INTEL, 0x10F6), board_82574}
   and to hw.h
        #define E1000_DEV_ID_82574L_NEW 0x10F6
  • Installed GA_pipeline 1.4
    • Install the Ubuntu Development Package:
     apt-get install build-essential
    • Installed fftw-3.2.1, need to compile with --enable-single;
    • Installed gnuplot;
    • Installed ImageMagik (for the convert function);
    • Installed zlib, bzlib2;
    • Install XML::Simple perl module.
    • Run make, all prerequisites passed.
    • Installed UNAFold (http://dinamelt.bioinfo.rpi.edu/) in /usr/local/

TSCC cluster

  • UPDATE: Triton is expected to go down by the start of July 2013. Home files will be moved to the TSCC servers and /project will be remounted on the TSCC servers (warning: uid/gid might be changed on project drive!)
  • INFO on data transfer and storage on TSCC: http://rci.ucsd.edu/computing/storage/data-transfer.html
  • INFO on starting jobs on TSCC: http://rci.ucsd.edu/computing/jobs/index.html
  • The Zhang lab current has four "home" nodes each having 16 cores and 64Gb of RAM. This is the best option when dealing with a large amount of data, such sequencing reads from one or more HiSeq runs. To use these nodes, you will need your own accounts at SDSC TSCC cluster (tscc-login.sdsc.edu). Once you logging in your account, you can create job files, and submit them to the "home" queue:
 qsub -q home-k4zhang job_name
 Or use the following line in the job file:
 #PBS -A k4zhang-group
 #PBS -q home-k4zhang
  • Our home nodes IDs are: tscc-2-33, tscc-2-35, tscc-2-37, tscc-2-39
  • The cheapest nodes for running jobs are: glean (runs free, no time limit), home-k4zhang (run on purchased time, no time limit), condo (run on purchased time, 8 hrs limit).
  • The pay-as-you-go nodes (72 hours limit) for running jobs are: hotel (can specify GPU), and pdafm (max 512GB memory, specify less fewer processors/memory to get charged less)
  • Node recommendations:
 High priority jobs should be run with home-k4zhang or condo.
      When running home-k4zhang, we can only start a limited number of jobs. Also, run qstat to see that no one in our lab is using glean
      To start many high priority jobs which takes less than 8 hours to complete, use condo
 Intermediate to low priority can be run with glean, there is unlimited time but may get kicked off at any time.
      Run glean on our nodes to lower the risks of getting kicked off, use any of the following lines to specify which of our nodes to run on:
      #PBS -l nodes=tscc-2-33
      #PBS -l nodes=tscc-2-35
      #PBS -l nodes=tscc-2-37
      #PBS -l nodes=tscc-2-39
      The only times we would get kicked off glean would be when someone in our lab starts a job with home-k4zhang or when condo is at full capacity (very unlikely!)
  • File storage and access:
 Do not transfer large files with scp, install and use bbftp and start a job to transfer files.
 (
 Large files for multiple uses can be transferred to /oasis (it have faster read/write than on /project or /home)!
 Small or large files for one-time use can be transferred to the path provided by the TMPDIR environment variable (this will copy the file to a local scratch space) then immediately processed. 
      Once the job finishes, any file in this local scratch space will be lost.
 Always copy results back to our lab server to avoid losing them.
 Avoid as much as possible reading and writing to /project or /home. Use /oasis instead.

Printers

  • Brother HL-5250DN, PFBH406, attached to Vaio-Z540 through USB. IP=192.168.1.24.
    • Used to be attached to ZhangLab-VGN through USB. IP=132.239.135.123. Before that, attached to ZhangLab-GelDoc through USB. IP=132.239.25.45
    • Can access via smb://192.168.1.24/BHL5250DN on Macs.
  • Brother HL-4150CN, PFBH402, network printer, IP=172.16.25.87.
  • Brother MFC-7220, PFBH402, attached to Time Capsule through USB.

Share computers

  • ZhangLab-GelDoc, PC desktop, attached to the BioRad GelDocXL system.
    • IP: 132.239.25.45.
    • To map the shared data folder, use the network path: \\132.239.25.45\GelImages
    • To map the shared data folder on mac, use the network smb: \\132.239.25.45\GelImages
  • ZhangLab-Chromo4, PC desktop, attached to Biorad Chromo4 Thermalcycler.
    • IP: 132.239.135.43.
    • To map the shared data folder, use the network path: \\132.239.135.43\Users
  • ZhangLab-VGN, PC laptop.
  • ZhangLab-IX81, the WinXP workstation attached to the Olympus IX81 microscope. \\172.16.25.86 MAC address: 00-25-64-A8-9D-1C, the password is "GenomeClub".

Biogem Data

  • To obtain data from Biogem, you must have sudo privileges.
  • On genome-miner, issue the command:
sudo mount /media/Biogem
  • To unmount the drive once you are done, type:
sudo umount /media/Biogem

Synology NAS - Diskstation

  • Synology DS2411+

Install Notes

  • Installed Synology Diskstation with 6 Hitachi 3 TB Drives
    • Part numbers: 0S03230 (Bays 1-3) and 0F12460 (Bays 4-6)
    • Seems to be the same drive, but from different production cycles?
    • Diskstation recognizes these as the same drive model
  • Installed Synology DSM Software using included disk
  • Currently formatting all six drives as a RAID5 - ~15 Tb storage space
    • Decided to perform full disk check to make sure we do not have any failures

Administering the NAS

  • Visit its IP Address (currently 192.168.1.22:5000, you must be on the ZhangLab wireless router) with any web browser
  • Username is admin, password is Zhang lab's common password
  • Can control disks, file sharing, and more from web interface

Installation Plans

Step 1

  • Copy SeqStore2 and other contents of Ext9T and Ext6T to 15 Tb partition
  • Erase Ext9T and Ext6T
  • Move the 3 functional 3TB drives from Ext9T to Diskstation
    • Set them up as either individual drives or RAID0 most likely (Scratch Space)
  • Load a backup 2TB drive into Ext9T
  • Re-setup each as either RAID5 or RAID0

Step 2

  • Dr. Zhang has ordered 5 more 3TB hard drives - load 3 of them into Diskstation
    • Set them up as either individual drives or RAID1? (Use for more long-term storage

Step 3

  • Next, temporarily back up Ext4T to Diskstation
  • Erase Ext4T, remove the two drives
  • Load two 3TB drives into Ext4T, set up as a 6 TB RAID0
  • Move data back from Diskstation to Ext4T

Results

  • This will leave us with:
    • Synology Diskstation: 15 TB RAID5, 18 TB individual disks, joined, or RAID0
    • Ext9T: 6-8 TB RAID5 or RAID0
    • Ext6T: 6-8 TB RAID5 or RAID0
    • Ext4T: 6 TB RAID0
    • Internal Storage on Genome-Miner: 8 TB in 4 disks
  • This will leave us with 3 2TB hard drives left over for backups.
    • Should order more Hitachi drives of this model; can use them as hotswaps when RAID fails


Synology NAS DiskStation (new)

Install Notes

  • Model Name: DS2413+
  • Installed at 02/01/2013
  • Location : PFBH 038 Confocal room

Network Information

  • MAC address
    • 00:11:32:19:59:EF
    • 00:11:32:19:59:F0
  • IP Address
    • 172.16.25.42(<-- Primary IP address)
    • 172.16.25.41(<-- Secondary IP address)
    • gateway 172.16.25.1
    • Subnetmask 255.255.255.0
    • nameserver 132.239.0.252
  • Server Name: ZhangLabNAS2
  • Access
    • bioeng16-25-42:5000 in any web browser (The given machine name of primary IP address from UCSD hostmaster is "bioeng16-25-42")
    • Username is admin, password is Zhang lab's common password

HDD Install and Volume management

  • 12 HDD have been installed (3TB, Toshiba).
  • Created volume as RAID5