Sunday, July 21, 2019

Quick Links: Software Application List

Application List and Shortcuts


The following are the list of applications that I use for systems administration, programming, and general use that may be useful for other persons. It is not an exhaustive list.

Disclaimer: This list is intended for use by Naresh Seegobin only!!! Use at your own risk!!!

No. Application Category Name Used For URL Comments
1 Utilities WinDirStat View Directory Structure . .
2 Utilities YumiLinux Multiboot USB Drives . .
3 Utilities FreeCommander File navigation . .
4 Utilities7-zip File compression and extraction . .
5 Utilities QuickPar/phpar2 File  . .
6 Optical Media Storage ImgBurn Burning to Optical media . .
7 Optical Media Storage DVDisaster Optical Media protection . .
8 Virtualisation VirtualBox Virtualisation . .
9 PDF Viewer Sumatra PDF Viewer . .
10 Terminal Utility Putty . . .
11 FTP FileZilla FTP cleint . .
12 Disk UtilityCrystalDisk Info Disk check . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .



Saturday, June 8, 2019

Net Admin Adventures in 2019 - Raspberry Pi as a DHCP and DNS server


Raspberry Pi as a DHCP and DNS server

Introduction

This blog article talks about migrating my DHCP server away from a wireless router and why you would want to setup a lightweight DHCP server on Linux.

Current Issues with the Existing ASUS RT-N16 Router

I was currently using an ASUS RT-N16 router with Tomato firmware for my wireless (multiple SSIDs), DHCP, DNS, quasi VPN, and internet router. The router was good for its time and is way past its end of life. Even though you can setup OpenVPN on it, the problems is that the NVRAM capacity is too small to handle multiple configurations. Some of these can be VPN certificates, static address assignments etc.

Firmware flashing is one thing but the NVRAM has a limited number of writes and any config or save done, even if it's just a character would mean a write of the entire NVRAM block. As NVRAM is flash memory ,each write to the NVRAM reduces its life.

Point to Note 1) The less changes you can make on your SOHO wireless router the better. As less writes will be made to your NVRAM. and will cause your router to last longer.


Once the NVRAM fails, resetting or firmware updating will not help. Unless you can change your NVRAM chip, best to recycle your router than to keep it as a paperweight. Maybe re-use the power adapter, antenna etc.


With these issues, I eventually migrated most services from this router. The VPN and firewall/internet router features were first migrated, then the wireless was replaced with an Ubiquiti AP and controller. DHCP was the last to migrate. Details of why it's the last will be explained later.

In addition to limited NVRAM capacity, the age or the router begins to show with the time it takes to boot up and start issuing IP addresses. Integrated Services Routers are great but with too many components and to keep costs down the components are cheaper resulting in a shorter life span than enterprise ISRs.

Slow boot up times means slow issuing of IP addresses, means devices cannot get access to the internet when they startup.

This is more for power failures, where once power is restored, all running devices will power on automatically. Even with a UPS, if the battery is drained and shuts down, the same process will happen as if there were no UPS. It is possible to add a delay when devices are powered on (a-la breakermatic devices), but that's for another blog post.

Either way devices boot up and start looking for an IP address. No IP from a DHCP server means APIPA assigned IP addresses or none at all. Statically assigned devices are great but it's difficult to do this for IoT.

Rule of thumb 1) for static IP addresses. ALWAYS have a reservation for these static IP devices. Preferably with their correct MAC addresses. Why? Should the devices' network configuration be reset, at least the IP address will be the same. Do this for VM's where their MAC addresses can change by the host if live migrations are done between hosts. Once migration is completed and the MAC address is changed, just update it in the reservation and keep a note of the previous MAC address.

Anyhow, back to getting a replacement wireless router, even though it's a good idea to get a replacement wireless router as it's newer, faster etc. the problems is that you will want to use it for more than just a DHCP server. And the more services you use on it, the more critical it becomes to your day to day operations, which means you would not want to restart it for maintenance, or if it fails (hardware or software).

Separating the services is a good idea at home or SOHO once the server hosting the service is low cost, manageable and lightweight.

Low cost is for obvious reasons, can't spent USD $200 on a simple DHCP server. You may want to then beef it up into a bigger server (RAM, storage), and fall into the trap of maximising the resources that you have.

Manageable, in that it doesn't take too much time to setup AND time to maintain it.

Lightweight in that it uses little resources in terms of CPU, Memory and disk space, maybe even low power consumption.


So the options are:
1) Windows DHCP server in a VM.
2) Windows Server in a VM with a third party DHCP server <-- don't know why you want to do this.
3) Windows DHCP server on a Micro PC (your USD $200 item)
4) Linux DHCP server (minimum or no GUI) in a VM.
5) Linux DHCP server (minimum or no GUI) on a MicroPC
6) Linux DHCP server on a single board ARM proc PC (e.g. Raspberry Pi).

Options 1) and 2) are easy to setup but are resource heavy (yeah, you say you can use Server Core or Nano) but management is too much for a home or SOHO. Then there are the licensing costs and the updates.

Option 3) is more that you will want to maximise your MicroPC.

Options 4) and 5) are great. No licensing costs, and the configurations are simple and stable.

Running in a VM there is the risk of the host failing AND the VMs may take too long to boot up. Then there is the chicken and egg scenario with the host waiting for the DHCP VM to boot up to lease IPs.

Rule of Thumb 2) For DHCP, get the fastest boot up device that can host this service.

DHCP is not supposed to be heavy. DNS maybe is but that's an added feature you can choose to use or not on the same device.

You may think a CISCO router can do the job, but tell me how long does it take to boot up?

You say a router should always be on backup power so restarts are minimal? What about your DHCP server? It can also be on backup power.

You say a router uses very little power? How about the RaspBerry Pi? is 10W of power too much?

So you see where this is going.

Option 6) is the best option so far. A low powered device that boots up fast and can run Linux.
Pi it is.

Why a Linux Alternative?

So why a Linux alternative to the router?
If you need to include optional information in your DHCP leases, unless the routers use a proper implementation of a Linux kernel/flavour you may not be able to add more options and additional features.

A lightweight version of Linux will boot fast.
It is highly customisable.
I need it to supply the following options:

dhcp-option=3,192.168.1.1
dhcp-option=5,192.168.1.1
domain=[DOMAIN-NAME]
Should you look at implementing a Linux based DHCP server in your enterprise network and need to add KMS server options in your DHCP leases use the following options in your config file:

srv-host=_vlmcs._tcp.[DOMAIN-NAME],[HOSTNAME1].[DOMAIN-NAME],1688,1
srv-host=_vlmcs._tcp.[DOMAIN-NAME],[HOSTNAME2].[DOMAIN-NAME],1688,2


More info on KMS and DNS can be found at Univ. Cornell's website, and Eric Ellis' blog.


Later on I would include next servers in my DHCP leases.

Btw, I tried using SOPHOS UTM for Home (https://www.sophos.com/en-us/products/free-tools/sophos-utm-home-edition.aspx) for DHCP but it doesn't have the ability to add these options and/or are not properly documented. Plus, as a UTM, its boot up time will be slow, (based on whatever HW it is running on) so not good for home with power failures and IoT. Actually it's boot up time is faster than my ageing ASUS RT-N16 router.


Which DHCP server service to use?
I prefer DNSMASQ (not DHCPD) is the best Linux DHCP server.
All the configurations are done via ssh and the modifications of the corresponding config file. DNSMASQ is highly customisable.

Even though DHCPD has a feature for load balancing DHCP servers, it doesn't work too well. At that point I didn't plan to setup or have an active passive failover implementation using DHCPD so I never bothered to continue to use it. DNSMASQ doesn't have load balancing (it would be a nice feature to implement in the future) but with config files, it can facilitate an active/passive implementation. That will be for another blog.

So a lightweight version of Linux running DNSMASQ on a Raspberry Pi is the best option for me.

Pi Problems

In my initial attempts to use a Raspberry Pi and Raspian was that the MAC address was always changing each time the Pi was restarted/power cycled. Not good if you want to set a reservation based on MAC address and setting a static IP address was also a pain. After days of searching, I found no definitive solution online, forum or even blog post. Plus, I didn't want to spend time researching a solution (and then blogging about it :)

So if I cannot set a static IP address, the Pi is worthless as a DHCP server. Project shelved for over a year.

Until a good friend introduced me to DietPi, https://www.dietpi.com/

Lighter than Raspbian and can actually configure a static IP address without hooking it up to a TV or monitor.

For my adventures in the DietPi, check this blog article out, "Net Admin Adventures in 2019 - DietPi on Raspberry Pi as DHCP and DNS server".

Summary

Ageing wireless routers have a limited lifespan due to their low cost components. The NVRAM is the most affected with its limited writes.
In selecting a replacement device, a Raspberry Pi can adequately function as a DHCP server once you can get Linux distribution to run on it allowing easy configuration of static IP address.

Step 1) Identify services to migrate.
Step 2) Identify your various hardware options, cost, configurability, maintenance and resource overhead.
Step 3) For each service, look at the various options that can be used. If it can be done on a low-powered device than a VM it may be the better option.
Step 4) Select your hardware/software solution and implement.

Point to Note 1) The less changes you can make on your SOHO wireless router the better. As less writes will be made to your NVRAM. and will cause your router to last longer.
Rule of thumb 1) for static IP addresses. ALWAYS have a reservation for these static IP devices. Preferably with their correct MAC addresses.
Rule of Thumb 2) For DHCP, get the fastest boot up device that can host this service.

Read my blog "Net Admin Adventures in 2019 - DietPi on Raspberry Pi as DHCP and DNS server" for the next steps on the Raspberry Pi.

Naresh
2019/06/08

Saturday, August 16, 2014

Oracle 12c on Oracle Linux 6.5 on Hyper-V (Server 2012R2 Updated) on Storage Spaces with De-duplication

 What more do you need.... :)


The following are for those who encountered some issues with installing Oracle Linux 6.5 on Hyper-V in the hopes of installing the Oracle 12c database on top of Oracle Linux 6.5.
Though Oracle 12c on Oracle Linux 6.5 is not supported on Hyper-V by Oracle, it may be necessary for the purpose of demonstrations and proof of concept.


Configuration environment:
  1. Server 2012R2 OS With GUI Updated
  2. Storage spaces enabled on,
  3. 4 drives in mirror mode
  4. De-Duplication enabled with 0 days schedule time (always applying de-duplication algorithms on all files)
Hyper-V Environment:
  1. OS with 4GB RAM
  2. 4 core CPU and set with maximum compatibility for live migration
  3. Boot disk, IDE, 320 GB
  4. Additional SCSI disk, 320 GB for u01 partition



Major symptoms encountered:
  1. Installation of Oracle Linux 6.5 went well (partition layout of /boot, 16GB SWAP, rest as / partition as default of ext4) but on reboot freezes on loading some modules.
  2. Some cases, loading completes and user able to login with root (now the real problem happens)
  3. Oracle Linux 6.5 OS keeps on using a lot of memory, giving up to 12GB RAM will still result in its using the swap partition.
  4. Attempting to format SCSI 320 GB disk (/dev/sdb) as MBR and ext4.
    1. Mounting works but cannot browse using Gnome explorer, nor ls via terminal.
    2. Un-mounting operation freezes, cannot umount drive.
    3. Shutting down OS will wait at un-mounting file systems.
    4. Force power down is necessary.

Could it be a configuration issue?








Mitigation Step 1 Attempted:
  1. Using kickstart script from oracle dev days VM's
  2. Formatting and mounting works but cannot browse using Gnome explorer, nor ls via terminal.
  3. Un-mounting operation freezes, cannot umount drive.
  4. Shutting down OS will wait at un-mounting file systems.
  5. Force power down is necessary.
 Same Symptoms Encountered:
  1. mkdir /u01/* failed
  2. umount /u01 failed
  3. repeated attempts of re-installation failed
  4. Shutting down OS will wait at un-mounting file systems
  5.  Force power down is necessary


Could it be  a storage spaces (and de-duplication) problem?




Mitigation Step 2 Attempted:
  1. Create VM on a host without de-duplication or storage spaces
  2. Oracle Linux used about 400MB RAM, no excessive usage of swap partition
  3. VM was configured with 2100 MB RAM
  4. Initial attempt worked with formatting and mounting of /dev/sb1 as /u01
  5. Gnome explorer worked, ls /u01 worke
 Same Symptoms Encountered:
  1. mkdir /u01/* failed
  2. umount /u01 failed
  3. repeated attempts of re-installation failed
  4. Shutting down OS will wait at un-mounting file systems
  5. Force power down is necessary

Could it be a disk IDE/SCSI problem?

Mitigation Step 3 Attempted:
  1. Create VM with both drives as IDE0 and IDE1
 Same Symptoms Encountered:
  1. mkdir /u01/* failed
  2. umount /u01 failed
  3. repeated attempts of re-installation failed
  4. Shutting down OS will wait at un-mounting file systems
  5. Force power down is necessary 



Beyond an IDE/SCSI disk problem?

 Mitigation Step 4 Attempted:

  1. Create VM with disk0 as IDE and disk1 as SCSI 
  2. Installed as normal but used ext3 partitions

Final Result (What everyone is waiting for!!!):
  1. Installation of Oracle Linux 6.5 worked
  2. Formatting, mounting of /dev/sdb1 as /u01 worked
  3. mkdir /u01/temp worked
  4. unmounting worked
  5. reboot worked.


At this point installation of Oracle 12c was successful....

 -------------------------------------------------------------------------------------------------------------------

Conclusion (IMHO)

The implemetation of ext4 on top of Hyper-V may be an issue causing disk processes to be in a busy wait.

Using ext3 worked as it has less resilient features implemented as compared to ext4.


Further tests:
xfs, btrfs etc...




Googling for a solution at this time (2014/08/10) did not produce any valid results but if anyone has encountered something similar with a working solution please share your experiences by posting in the comments or pm me.














Sunday, October 28, 2012

Windows batch script for creating hardlinks in one folder for collection ISO files in seperate folders for your YUMI live linux installer

So we want to create a multiboot USB flashdrive with our favourite Linux distribution. You download the YUMI windows installer at http://www.pendrivelinux.com/yumi-multiboot-usb-creator/ and start creating your multiboot Linux flashdrives.

It is a very nice application to use, and it can even download the ISO files for you. All well and good for a small flashdrive. But what if you want to keeps these ISOs for later for a larger flashdrive or for an external HDD? And what if you want to download a whole lot of ISOs? YUMI installer will allow you to do this one by one which is not bad.
So after the first 10 ISO's you say hey, might as well sort them into their correct distributions with directories. So after a few hundred GB of downloads and sorting you have a nice collection of Linux distributions. But now you want to have all these ISO files in onle single folder for the YUMI application to detect your ISOs automatically.

What do you do?

We have some possible solutions:

Solution One: Perform a dir search of all ISO files and copy them to a single directory which takes up space but it is doable and scriptable

Solution Two: Forget the nice directory arrangement and move all the files to a single directory

Solution Three: Create hardlinks to the files to a single directory

 Solution three is what I wanted. I have searched for something that I can use in Windows but decided to cobble together a script.

I created a folder named Linux-ISO-HardLinks which is a seperate folder from my collection but on the same drive. You can use whatever name that you want for the folder. Make sure you know that this folder contains hard links. It starts to become confusing as the file links are indistinguishable from the originals. Deleting a hardlink (by deleting the file name) sets the counter down by one, http://en.wikipedia.org/wiki/NTFS_symbolic_link has more info on this. If there are no other hardlinks, deleting the file will actually delete it, so be careful.

A batch file named CreateLinuxISOHardLinks.bat (can also use your own filename) and set the attribute to system (more on this later):

echo "" > CreateLinuxISOHardLinks.bat
attrib +S CreateLinuxISOHardLinks.bat



So my directory structure will look like this (uing the "tree" command and some copy and pasting into notepad):

├───Linux
│ ├───Acronis Antimalware CD
│ │ AcronisAntimalwareScanCD.iso
│ ├───AntiX
│ │ antiX-12-486.iso
│ └───archlinux
│      archlinux-2012.10.06-dual.iso
└───Linux-ISO-HardLink
     CreateLinuxISOHardLinks.bat

 And the script for the hardlink creation is below:

  1. del /A:-S *.* /Q
  2. for /f "delims=" %%a in ('Dir /s /b ..\Linux\*.ISO') do echo fsutil hardlink create "%%~nxa" "..\..\..%%~pa%%~nxa" >>output.txt & fsutil hardlink create "%%~nxa" "..\..\..%%~pa%%~nxa" 1>> output.txt 2>>&1
  3. copy ..\YUMI-0.0.7.7.exe .\


The first line deletes all the current hard link files except for your batch file which is a system file (which we had set earlier on). We want to delete the files to clean up the directory and start from fresh. Remember, if we delete the source ISO files in your Linux folder, the files will still exist until this folder is cleaned. Be very careful with this!!!!

The second line is the core hardlink generation script. Inside the for loop, a dir is run for all the .ISO files in the Linux directory. This command returns a list of the full path of each ISO file.

  1. 'Dir /s /b ..\Linux\*.ISO'

So for each result in our dir comamnd we execute the commands in the do part of the loop. Firstly we echo the command to be run to an output file (for debugging purposes).

  1. do echo fsutil hardlink create "%%~nxa" "..\..\..%%~pa%%~nxa" >>output.txt

Followed by an & symbol to seperate the commands, we then execute the hard link command with outputs from both stdout and stderr to the same outputfile.

  1. & fsutil hardlink create "%%~nxa" "..\..\..%%~pa%%~nxa" 1>> output.txt 2>>&1
The regular expressions "%%~nxa" is the name of the file alone and the "..\..\..%%~pa%%~nxa" is a manipulation of relative paths to get the actual file to link to (Remember the directory structure above).

The neat thing is that the do command is run for each result of the dir command thus putting the correct sequence of events into your output file for debugging. Works well for duplicate ISO files which you can clean up later on. I tried this on an external HDD which worked well.


Other Uses for this script with modifications:

This can be used to take your entire movie or music collection and place them in a single folder for sharing or streaming. http://www.mede8erforum.com/index.php?topic=5335.5;wap2

Summary:

Hardlinking solves the space problem instead of duplicating files.


***** DISCLAIMER *****
Do this at your own risk!!! Do not try this on a drive that you are not so sure of nor on your system drive. The delete command is the most dangerous so be careful of the paths that you are using!!!


Some references that I used to create the script in addition to the for /? command:
http://en.wikipedia.org/wiki/NTFS_symbolic_link
http://www.microsoft.com/resources/documentation/windows/xp/all/proddocs/en-us/fsutil_hardlink.mspx?mfr=true
http://superuser.com/questions/344155/how-do-i-recursively-list-filenames-only-in-dos-windows
http://stackoverflow.com/questions/1420965/redirect-stdout-and-stderr-to-a-single-file-in-dos

Sunday, November 27, 2011

About this blog...

This is my first blog entry ever... :)
So a few FAQ's for those interested.
 DISCLAIMER: The views expressed by the author are of his opinion and not of anyone else.
If there are any problems, a quick pm can help resolve almost any issue.


Why this blog?

The internet is a rich source of information, I have a saying that there are so many people in the world that there is a chance that someone, somewhere may have had a similar problem that you have and would have posted a possible solution out there in the form of support websites, Wikipedia, blogs, Google and of course RTFM.

But then RTFM, Google and blog website may not provide the answers that you need and you will have to plough through tens of pages and sites to formulate a solution suitable for your needs.

So it comes down to why this site? In the event that this ploughing and concocting provides something useful to more than just me then it will be a nice idea to share with the community. And possibly draw comments from other professionals to improve the solution, which in turn will benefit everyone....


Why this title?

nucleo pauoris or a kernel panic. I wanted something technology related linking with Latin.

Why use Latin?

Well the English kernelpanic was already taken :)
Just kidding, Latin is a fascinating language that we use even today, and in my opinion it is NOT a dead language, or else we would have stopped using Latin to name new species and convert the current names to whatever language we want, English I guess or maybe French. 
But that's just me... we will see how it goes.
I do not speak Latin, it is just fascinating both scientifically and historically.

What's this blog about?

A little more to the point, it is a blog about solutions to IT related problems that I have encountered. I will try my best to keep it detailed enough so that it can be beneficial.
So no nonsensical ramblings like, the latest Linux flavor of the month looks great, or check out this bleeding edge piece of technology. All these might be in another blog :)


 Why  blogger?

At the moment, it is the fastest to setup and it is linked with my Gmail account.
Later on I will open up other blogs on other popular sites like WordPress etc.
In terms of the future of Blogger, well once Google exist, blogger will remain, unless they do something similar to what they did to Google wave and the Google groups.