Lab IT

From ZhangLabWiki
Jump to navigation Jump to search

Register a new computer to the campus network

Web-Based Monitoring and Alarm Notification Web600

  • [1]
  • Mac Address: 00:07:F9:00:58:83
  • Plugged into Port 4.1.34 B in Zhang main lab
  • 132.239.25.73
  • gateway 132.239.25.1
  • netmask 255.255.255.0
  • nameserver 132.239.0.252
  • admin:Zhang lab's common password

NAS

  • Apple Time Capsule 877b54
    • MAC address: 00:1F:00:1F:5B:87:7B:54
    • Wireless password: Zhang lab's common password.
    • Backup of lab wiki, also serves as a wireless router.

Lab website (WordPress)

  • Create a MySQL database:
  /opt/local/lib/mysql5/bin/mysql -u root -p
   create database ZhangLab_WP_DB;
   create user 'WP_user'@'localhost' identified by '7d7c65LzW4';
   GRANT ALL PRIVILEGES ON ZhangLab_WP_DB.* TO 'WP_user'@'localhost';
   FLUSH PRIVILEGES;
  • Download wordpress-4.5.3.tar.gz, uncompress, and edit wp-config.php.
  • Copy the wordpress folder to the web server directory.

Wiki server

  cd /etc/httpd/
  sudo htpasswd passwords new_user_name
  
  Edit the httpd.conf file.
  Add the new_user_name to the following line:
  Require user kunzhang ...some_existing_user_name
      • Wiki backup
  Launchd deamon:  /Library/LaunchDeamons/labwiki.backup
  Script for backup: /Library/Scripts/wiki_backup.sh
  • Server migration from Mac Mini to Mac Pro: Migration Log
  • Reads from genome-miner are shared in /Volumes/genome-miner-SeqStore2/ (See [2])
  • MediaWiki was upgrade from v 1.13.2 to v 1.15.1
    apachectl stop
    #backup the mysql databases
    mv LabNotes  LabNotes.v1.13.2
    tar -xzvf mediawiki-1.15.1.tar.gz
    mv mediawiki-1.15.1 LabNotes
    cp LabNotes.v1.13.2/LocalSettings.php LabNotes
    cp -r LabNotes.v1.13.2/extensions LabNotes
    cp /LabNotes/AdminSettings.sample /LabNotes/AdminSettings.php
    #edit the /LabNotes/AdminSettings.php file for the admin previlege
    cd LabNotes/maintenance
    php update.php --aconf ../AdminSettings.php
    cd ../..
    mv LabNotes.v1.13.2/upload LabNotes
    mv LabNotes.v1.13.2/zhangloupload LabNotes
    apachectl start
  • Sphinx 0.9.9 full text search engine installed (Jan-23-2010). Updated on 10-Apr-2013 (/Users/kunzhang/www/LabNotes/extensions/SphinxSearch).
  • MediaWiki was upgrade from v 1.15.1 to v 1.18.2 (April-08-2012)
    mv LabNotes  LabNotes.v1.13.2
    tar -xzvf mediawiki-1.18.2.tar.gz
    mv mediawiki-1.18.1 LabNotes.v1.18.2
    cp LabNotes/LocalSettings.php LabNotes.v1.18.2
    cp -r LabNotes.v1.13.2/extensions LabNotes
    cd LabNotes.v1.18.2/maintenance
    php update.php
    sudo mv LabNotes/upload LabNotes.v1.18.2
    sudo mv LabNotes/zhangloupload LabNotes.v1.18.2
    sudo mv probedesign ../LabNotes.v1.18.2/
    sudo mv PeakPicker.jar ../LabNotes.v1.18.2/
    sudo mv SeqScannerInstall.exe ../LabNotes.v1.18.2/  
    sudo mv RobertsLabNotesSupplement ../LabNotes.v1.18.2/ 
    mv LabNotes  LabNotes.v1.15.1
    mv LabNotes.v1.18.2  LabNotes
    download the latest version of Sphinx extension: SphinxSearch-MW1.18-r92378.tar.gz 
    Extract the tarball and copy all the files to LabNote/extension/SphinxSearch folder.
    To rebuild the mysql database from mysql dump:
      /usr/local/mysql/bin/mysql -u wikiuser -pPASSWORD wikidb < wiki_db_current.sql
    Update on 2/6/2013
    I installed the latest version of mysql5 using MacPorts
      sudo /opt/local/bin/port install mysql5-server
    Note that the mysql files were installed at /opt/local/bin
    To start the MySQL server on launch, 
      sudo launchctl load -w /Library/LaunchDaemons/org.macports.mysql5.plist

Meangenemachine

  • Duel Intel Xeon X5645 (12 cores), 32 GB RAM
  • Booting from 500GB drive
  • Drive space: 600GB_store, 1TB_store1, 1TB_store2, 2TB_store1, 2TB_store2
  • Ubuntu 11.04
  • For general use
        • address: meangenemachine.dynamic.ucsd.edu
  • Setting up DHCP network:
    • Request DHCP-only networking with ACS
    • In: /etc/network/interfaces
auto lo eth0
iface eth0 inet dhcp
iface lo inet loopback
    • Restart networking
sudo /etc/init.d/networking restart
    • Get IP
sudo dhclient

Genemapster

  • Dual Intel Xeon X5520 (8 cores), 32 GB RAM, Tesla C1060 GPU
  • Ubuntu 10.04.3 LTS
  • For general use and for running GPU softwares
        • address: genemapster.dynamic.ucsd.edu

Genome-miner

Data storage

Storage Location Total size RAID? Time of storage Ave. write speed (MB/s) Ave. read speed (MB/s)
/media/TmpStore1 6 TB None Analysis time/Ref Data 145.3 136.3
/media/TmpStore2 6TB None Analysis time/Ref Data 26.1 105.8
/media/Ext4T 4T None Ref Data 80.6 100.3
/media/Ext12T 12T None Analysis time/Ref Data 161.0 76.3
/media/Syn_15T 15T RAID5 INF 94.8 82.7
/media/LTS_15T 15T RAID5 INF 95.1 77.7
/media/LTS_33T 33T RAID5 INF 13.5 41.3
/media/LTS_60T 60T RAID5 INF 96.2 108.0
  • Recently reinstalled using MacOS instead of Ubuntu; see notes here
  • A MacPro (8 core, 16GB RAM, 4TB), dedicated for GA Pipeline (PFBH406)
    • MAC addresses:
      • Port 1: 00:25:00:ee:6f:c8; connected to the campus network;
        • IP=132.239.135.41;
        • Subnet: 255.255.0.0
        • Gateway: 132.239.135.1
        • DNS: 132.239.0.252, 128.54.16.2
      • Port 2: 00:25:00:ee:3f:85; connected to GA PC;
        • IP=192.168.137.11
  • Installation of Ubuntu 9.04 (AMD64 Desktop).
    • Installed rEFIt for dual booting (http://refit.sourceforge.net/).
    • Installed Ubuntu 9.04 Desktop from ISO image.
      • Turned out that all the hard drives were not recognized with the RAID card. Removed the RAID card.
      • Press "c" to boot from the CD.
      • The OS was install in the hard drive in Bay 4.
      • The other two hard drives in Bay 2&3 were also reformatted to ext3
    • Ubuntu booted up correctly, but without network connection. Turned out that the Ethernet card was too new.
      • Downloaded the driver from sf.net/projects/e1000, manually changed the following two files, compiled and installed the driver.
   Added the following to the netdev.c: 
        {PCI_VDEVICE(INTEL, 0x10F6), board_82574}
   and to hw.h
        #define E1000_DEV_ID_82574L_NEW 0x10F6
  • Installed GA_pipeline 1.4
    • Install the Ubuntu Development Package:
     apt-get install build-essential
    • Installed fftw-3.2.1, need to compile with --enable-single;
    • Installed gnuplot;
    • Installed ImageMagik (for the convert function);
    • Installed zlib, bzlib2;
    • Install XML::Simple perl module.
    • Run make, all prerequisites passed.
    • Installed UNAFold (http://dinamelt.bioinfo.rpi.edu/) in /usr/local/

TSCC cluster

  • UPDATE: Triton is expected to go down by the start of July 2013. Home files will be moved to the TSCC servers and /project will be remounted on the TSCC servers (warning: uid/gid might be changed on project drive!)
  • INFO on data transfer and storage on TSCC: http://rci.ucsd.edu/computing/storage/data-transfer.html
  • INFO on starting jobs on TSCC: http://rci.ucsd.edu/computing/jobs/index.html
  • The Zhang lab current has four "home" nodes each having 16 cores and 64Gb of RAM. This is the best option when dealing with a large amount of data, such sequencing reads from one or more HiSeq runs. To use these nodes, you will need your own accounts at SDSC TSCC cluster (tscc-login.sdsc.edu). Once you logging in your account, you can create job files, and submit them to the queue:
 [#PBS -A k4zhang-group] - this line may not need to be in job file to submit to home, condo, or glean.
 Then submit job to a specific queue with
 Submit to home queue: qsub -q home-k4zhang job_name 
 Submit to condo queue: qsub -q condo job_name -W group_list=condo-group
 Submit to glean queue: qsub -q glean job_name -W group_list=condo-group
 Submit to hotel queue (pay-per-use): qsub -q hotel job_name
 Submit to pdafm queue (pay-per-use): qsub -q pdafm job_name
  • Our home nodes IDs are: tscc-2-33, tscc-2-35, tscc-2-37, tscc-2-39
  • The cheapest nodes for running jobs are: glean (runs free, no time limit), home-k4zhang (run on purchased time, no time limit), condo (run on purchased time, 8 hrs limit).
  • The pay-as-you-go nodes (72 hours limit) for running jobs are: hotel (can specify GPU), and pdafm (max 512GB memory, specify less fewer processors/memory to get charged less)
  • Node recommendations:
 High priority jobs should be run with home-k4zhang or condo.
      When running home-k4zhang, we can only start a limited number of jobs. Also, run qstat to see that no one in our lab is using glean
      To start many high priority jobs which takes less than 8 hours to complete, use condo
 Intermediate to low priority can be run with glean, there is unlimited time but may get kicked off at any time.
      Run glean on our nodes to lower the risks of getting kicked off, use any of the following lines to specify which of our nodes to run on:
      #PBS -l nodes=tscc-2-33:ppn=X
      #PBS -l nodes=tscc-2-35:ppn=X
      #PBS -l nodes=tscc-2-37:ppn=X
      #PBS -l nodes=tscc-2-39:ppn=X
      (X = number of processors, if not specified, the entire node might be used and we can't start more than one jobs on each node.)
      The only times we would get kicked off glean would be when someone in our lab starts a job with home-k4zhang or when condo is at full capacity (very unlikely!)
  • File storage and access:
 Do not transfer large files with scp, install and use bbftp and start a job to transfer files.
 (
 Large files for multiple uses can be transferred to /oasis (it have faster read/write than on /project or /home)!
 Small or large files for one-time use can be transferred to the path provided by the TMPDIR environment variable (this will copy the file to a local scratch space) then immediately processed. 
      Once the job finishes, any file in this local scratch space will be lost.
 Always copy results back to our lab server to avoid losing them.
 Avoid as much as possible reading and writing to /project or /home. Use /oasis instead.
  • Checking the output of a job while the job is still running:
    1. Use qstat -u [your_username_here] to get the job id: should look something like "XXXXXXX.tscc-mgr.local" where the 7 digits before ".tscc-mgr.local" are your job ID.
    2. Use checkjob [your_job_id_here] to get info about your job. The node your job is running on should be under "Allocated Nodes" in the format tscc-x-XX:YY where the two digits after the colon are the number of processors requested.
    3. ssh into the node your job is using "ssh tscc-x-XX"
    4. cd /var/spool/torque/spool
    5. There should be two files associated with your job id labeled something like "XXXXXXX.OU" and "XXXXXXX.ER". These are the stdout and stderr files, respectively and you can view them to check the output of your job. This is a convenient way to check job progress

Printers

  • Brother HL-5250DN, PFBH406, attached to Vaio-Z540 through USB. IP=192.168.1.24.
    • Used to be attached to ZhangLab-VGN through USB. IP=132.239.135.123. Before that, attached to ZhangLab-GelDoc through USB. IP=132.239.25.45
    • Can access via smb://192.168.1.24/BHL5250DN on Macs.
  • Brother HL-4150CN, PFBH402, network printer, IP=172.16.25.87.
  • Brother MFC-7220, PFBH402, attached to Time Capsule through USB.
  • Brother HL-L2340D, PFBH406, network printer, IP=192.168.1.22. Access through ZhangLab network.

Share computers

  • ZhangLab-GelDoc, PC desktop, attached to the BioRad GelDocXL system.
    • IP: 132.239.25.45.
    • To map the shared data folder, use the network path: \\132.239.25.45\GelImages
    • To map the shared data folder on mac, use the network smb: \\132.239.25.45\GelImages
  • ZhangLab-Chromo4, PC desktop, attached to Biorad Chromo4 Thermalcycler.
    • IP: 132.239.135.43.
    • To map the shared data folder, use the network path: \\132.239.135.43\Users
  • ZhangLab-VGN, PC laptop.
  • ZhangLab-IX81, the WinXP workstation attached to the Olympus IX81 microscope. \\172.16.25.86 MAC address: 00-25-64-A8-9D-1C, the password is "GenomeClub".

Biogem Data

  • To obtain data from Biogem, you must have sudo privileges.
  • On genome-miner, issue the command:
sudo mount /media/Biogem
  • To unmount the drive once you are done, type:
sudo umount /media/Biogem

RAID Enclosures

  • Buffalo DriveStation Quattro™ - HD-QSSU2/R5
    • Ext4T. Maximum capacity is 8TB (4x2TB)
  • Other World Computing (OWC) Mercury Elite Pro Qx2 RAID
    • Ext12T

Mounting the RAID Enclosures

  • Athurva's notes (October 29th, 2014):
    • The particular version of Linux on genome-miner sometimes has trouble scanning large drives with mount. You can use gparted to scan the drives, and then the mount command will work.
    • If you launch gparted, go to the troublesome drive, right click, choose manage flags, uncheck and then recheck the raid flag, and then hit ok, the drive will get rescanned and mount commands will then work from the command line.

Synology NAS - Diskstation

  • Synology DS2411+ (DiskStation)
  • Synology DS2413+ (ZhangLabNAS2)
  • Synology DS2415+ (ZhangLabNAS3)

Install Notes

  • Installed Synology Diskstation with 6 Hitachi 3 TB Drives
    • Part numbers: 0S03230 (Bays 1-3) and 0F12460 (Bays 4-6)
    • Seems to be the same drive, but from different production cycles?
    • Diskstation recognizes these as the same drive model
  • Installed Synology DSM Software using included disk
  • Currently formatting all six drives as a RAID5 - ~15 Tb storage space
    • Decided to perform full disk check to make sure we do not have any failures

Administering the NAS

  • Visit its IP Address (currently 192.168.1.22:5000, you must be on the ZhangLab wireless router) with any web browser
  • Username is admin, password is Zhang lab's common password
  • Can control disks, file sharing, and more from web interface

Accessing the NAS

  • October 29, 2014:
    • Can log onto the Synology via 192.168.137.5:5000 on genome-miner using zhanglab and the Zhang lab's common password.
    • Athurva's command example to mount Synology drives on genome-miner:
 sudo mount.cifs \\\\192.168.137.5\\Syn_15T /media/Syn_15T -o username=ZhangLab,dir_mode=0775,uid=ajgore,gid=sambashare,noperm
 sudo mount.cifs \\\\192.168.137.5\\LTS_15T /media/LTS_15T -o username=ZhangLab,dir_mode=0775,uid=ajgore,gid=sambashare,noperm
 sudo mount.cifs \\\\192.168.137.5\\LTS_33T /media/LTS_33T -o username=ZhangLab,dir_mode=0775,uid=ajgore,gid=sambashare,noperm
    • Athurva's mounts:
 //192.168.137.5/LTS_15T/    ...  ...  ...  68% /media/LTS_15T
 //192.168.137.5/LTS_33T/    ...  ...  ...  69% /media/LTS_33T
 //192.168.137.5/Syn15T/     ...  ...  ...  23% /media/Syn_15T

Installation Plans

Step 1

  • Copy SeqStore2 and other contents of Ext9T and Ext6T to 15 Tb partition
  • Erase Ext9T and Ext6T
  • Move the 3 functional 3TB drives from Ext9T to Diskstation
    • Set them up as either individual drives or RAID0 most likely (Scratch Space)
  • Load a backup 2TB drive into Ext9T
  • Re-setup each as either RAID5 or RAID0

Step 2

  • Dr. Zhang has ordered 5 more 3TB hard drives - load 3 of them into Diskstation
    • Set them up as either individual drives or RAID1? (Use for more long-term storage

Step 3

  • Next, temporarily back up Ext4T to Diskstation
  • Erase Ext4T, remove the two drives
  • Load two 3TB drives into Ext4T, set up as a 6 TB RAID0
  • Move data back from Diskstation to Ext4T

Results

  • This will leave us with:
    • Synology Diskstation: 15 TB RAID5, 18 TB individual disks, joined, or RAID0
    • Ext9T: 6-8 TB RAID5 or RAID0
    • Ext6T: 6-8 TB RAID5 or RAID0
    • Ext4T: 6 TB RAID0
    • Internal Storage on Genome-Miner: 8 TB in 4 disks
  • This will leave us with 3 2TB hard drives left over for backups.
    • Should order more Hitachi drives of this model; can use them as hotswaps when RAID fails


Synology NAS DiskStation (NAS2 in the microscope room)

Install Notes

  • Model Name: DS2413+
  • Installed at 02/01/2013
  • Location : PFBH 038 Confocal room

Network Information

  • Updated: 1/7/2016 - Due to volume crash in 12/2015 had to change IP address for Synology to remote access
  • MAC address
    • 00:11:32:19:59:EF
    • 00:11:32:19:59:F0
  • IP Address
    • 132.239.25.19
    • gateway 132.239.25.1
    • Subnetmask 255.255.255.0
    • nameserver 132.239.0.252 (not sure about this)
  • Server Name: ZhangLabNAS2
  • Access
    • bioeng-25-19:5000 (or http://132.239.25.19:5000) in any web browser (The given machine name of primary IP address from UCSD hostmaster is "bioeng-25-19")
    • Username is admin, password is Zhang lab's common password (capital G and C)
      • 2 failed login attempts within 10 min will lock you out for a day

HDD Install and Volume management

  • 12 HDD have been installed (3TB each, Toshiba, total 36TB, --> 30TB available + 6TB backup).
  • Created volume as RAID5

Synology NAS DiskStation (NAS3 in sequencing room)

Install Notes

  • Model Name: DS2415+, 12 X 6TB drives
  • Installed at 10/17/2015
  • Location : Gene Sequencing Room, connected to NetGear Switch
  • Drive mount command:
 sudo mount.cifs \\\\192.168.137.6\\LTS_60T /media/LTS_60T -o username=ZhangLab,dir_mode=0775,uid=ajgore,gid=sambashare,noperm
  • Installation guide:
    • (1) Plug all the drives in.
    • (2) Connect at least one ethernet connection from new NAS to NetGear switch, and second ethernet connection from NetGeat switch to a laptop. (Probably can't use Genome-miner unless we can disconnect Genome-miner from the wall jack). For the second ethernet connection, borrow the one connected to the GA IIx PC.
    • (3) Configure the network IPv4 to the same IP range and subnet. I used 192.168.137.10 and 255.255.255.0
    • (4) Download and install Synology Assistant. If you haven't already, download the .pat file for installing the latest DSM on the new Synology. Turn off wireless networking.
    • (5) Run Synology Assistant, should be able to see both the DiskStation and new Synology NAS
    • (6) Install and configure the network connection for the new NAS:
      • manually set the same IP range as the others, and subnet 255.255.255.0
    • (7) Once the new NAS is completely installed, log onto the NAS and set up a new Share Folder. Set up ZhangLab user account. Make sure guest account is disabled. Configure other settings to allow SSH, NFTP, file sharing, etc.

Network Information

  • IP Address
    • 192.168.137.6
    • 192.168.137.7
    • Default gateway: 192.168.137.1
    • Subnetmask: 255.255.255.0
    • DNS server: 192.168.137.1
  • Server Name: ZhangLabNAS3
  • Access
    • Only via the NetGear switch and must be in the same IP range.
    • Username is admin, password is Zhang lab's common password