[131067710010] |Processing bash variable with sed
[131067710020] |Been banging my head off a wall on this
[131067710030] |bash variable LATLNG contains a latitude &longitude value in brackets like so
[131067710040] |(53.3096,-6.28396)
[131067710050] |I want to parse these into a variable called LAT and LON which I'm trying to do via sed like so
[131067710060] |LAT=$(sed "s/(\(.*\),\(.*\))/\1/g" "$LATLNG") LON=$(sed "s/(\(.*\),\(.*\))/\2/g" "$LATLNG")
[131067710070] |However, I get the following error:
[131067710080] |sed: can't read (53.3096,-6.28396): No such file or directory
[131067710090] |I'm obviously doing something silly here but it's very late and I cant see what it is. I've tried several other variations without luck too.
[131067720010] |This can be solved via pure shell syntax.
[131067720020] |It does require a temp variable because of the parentheses (brackets) though:
[131067720030] |Alternatively, you can do it in one go by playing with IFS and using the read builtin:
[131067730010] |SiegeX's answer is better for this particular case, but you should also know how to pass arbitrary text to sed.
[131067730020] |sed is expecting filenames as its second, third, etc. parameters, and if it doesn't find any filenames, it reads from its standard input.
[131067730030] |So if you have text that you want to process that's not in a file, you have to pipe it to sed.
[131067730040] |The most straightforward way is this:
[131067730050] |So your example would become:
[131067730060] |Alternate (better but more obscure) Methods
[131067730070] |If you think there's any chance that $LATLNG could begin with a dash, or if you want to be pedantic, you should use printf instead of echo:
[131067730080] |Or a "here document", but that can be a little awkward with the construct you're using:
[131067730090] |Or if you're using bash and not worried about portability, you can use a "here string":
[131067740010] |Here's a solution that will work in any POSIX shell:
[131067740020] |Here's another equally portable solution that parses the specific syntax used.
[131067750010] |How to install from a Linux live CD that does not support installing?
[131067750020] |I tried out the linuX-gamers live DVD and like it so much that I want to have it as the main operating system on my desktop.
[131067750030] |The FAQ says:
[131067750040] |Can I install or copy the medium to my hard disk?
[131067750050] |No, the software is only designed to boot from a live medium.
[131067750060] |However, as the live DVD is said to be based on Arch Linux I think it is pretty much possible to put it onto the hard drive.
[131067750070] |A painful way to do this would be to install Arch and try to make it look like this DVD.
[131067750080] |The download page says that the ISO is isohybrid, I don't know if it makes any difference.
[131067750090] |Is there a (reliable) way to turn an ISO like that into a working installation?
[131067750100] |I wouldn't mind spending a few days to mess with it.
[131067760010] |You can use unetbootin to boot iso images from harddisk.
[131067760020] |Else the commands depend on your bootloader.
[131067760030] |Example: http://www.linuxquestions.org/questions/linux-general-1/boot-iso-image-from-hard-disk-294744
[131067770010] |I emailed the author of the distro and was told that dding the content of the ISO to the hard drive might work.
[131067770020] |So I'm all backed up and trying that right now.
[131067770030] |I think this makes an answer, whether it works or not.
[131067770040] |I will update this after seeing the result.
[131067770050] |Update: This actually works, to the extend that it gives me a working system on the local hard drive.
[131067770060] |The file system is volatile (all changes are gone after shutting down).
[131067770070] |The disk is left in an "unpartitioned, invalid" state so I don't think this counts as a full installation.
[131067780010] |You could test this by using Virtualbox.
[131067780020] |Also, the ISO is not likely to contain a hard disk fs, therefore you would need to copy files, not dd the raw data.
[131067780030] |Install Virtualbox, and get a Linux distro running on it.
[131067780040] |Then create another virtual hard drive, attach it to the Linux distro and partition it.
[131067780050] |Create a filesystem on the VHD such as ext3fs.
[131067780060] |Then attach the ISO to this VM, i.e. mount the CD, and copy everything to the partition that you created.
[131067780070] |tar cvf - /mountedcd | (cd /mountedVHD; tar xf - .)
[131067780080] |Shutdown this VM.
[131067780090] |Then create another virtual machine using the partitioned HD as the hard drive.
[131067780100] |When you start this VM it should boot up into the live distro except it is now on a hard drive.
[131067780110] |You will may need to do something to the partition before going much further, such as remount it read-write, or editing some of the init files.
[131067780120] |Write down everything you do, and then you can repeat it with a real system and real hard drive.
[131067790010] |As of today I have successfully installed this distribution and can use it as if it were Arch :) Below is the simplest way to do so:
[131067790020] |Install Arch on the hard drive
[131067790030] |Remove everything in / (in the local disk), except for /boot
[131067790040] |Mount the root-image.sqfs image in the linuX-gamers live DVD and copy everything inside to /
[131067790050] |Repeat the previous step with the overlay.sqfs image
[131067790060] |Step 2, 3, 4 may have to be performed with a live CD.
[131067790070] |Further customization is needed but the system can boot and function correctly after step 4.
[131067790080] |This answer is really specific to this live DVD and thus not applicable to other live CD/DVD.
[131067800010] |Emacs command to cut OR copy to system clipboard?
[131067800020] |As a very new Emacs user, I've been using Ctrl+w and Ctrl+y to cut/copy and paste between buffers in Emacs.
[131067800030] |Is there a way to do the same thing so that not just Emacs, but the entire system, can have access to what I am copying and pasting?
[131067800040] |Thanks.
[131067800050] |P.S.
[131067800060] |I would like to do this in Linux (in a terminal emulator running under X) and Mac OS X (e.g. with Terminal).
[131067810010] |In CarbonEmacs (a native look-n-feel build for Mac OS X) copy and paste between command-keys in native apps, middle-click in X and C-[w,y] in emacs just work.
[131067820010] |You must be talking about X clipboard, so put this line into your .emacs file:
[131067830010] |When emacs is running under X, there are a number of ways to implement this (if I understand you correctly).
[131067830020] |If you have a recent version of emacs, I think the following lines in .emacs creates the most intuitive behavior:
[131067830030] |As always, the poorly organized but informative EmacsWiki has information on this issue.
[131067830040] |However, when you are running in a terminal these settings (and the others mentioned so far) won't help you since the terminal is sitting between emacs and X. How to copy text from the terminal will usually be terminal dependent.
[131067840010] |Copy a file from a nautilus-script to clipboard
[131067840020] |I have a nautilus script that generates an archive file based on the files selected in the nautilus window.
[131067840030] |This archive file is created in the /tmp directory.
[131067840040] |I want a way to copy this file to the clipboard from the script, so that the user can just go to desktop or home directory and paste it.
[131067840050] |I have tried doing this with xclip and xsel, but they don't seem to replicate a file copying operation, rather they copy the contents of a file.
[131067840060] |Neither of them do what I need.
[131067840070] |So, I want to know if this is possible, and if it is, how should I go about it?
[131067840080] |Thanks.
[131067850010] |you should understand that when you 'copy' a file to the clipboard in nautilus, all you're actually doing is putting the absolute filepath on the clipboard (you can use a clipboard manager to verify this).
[131067850020] |So you simply need to have your script put the filepath on the clipboard, and you will achieve the result you're looking for.
[131067850030] |Try it!
[131067860010] |number of files containing a given string
[131067860020] |How to count the number of files (in a directory) containing into a given string as input in bash/sh? thanks
[131067870010] |Clarification: This looks for the number of files that has the "pattern_to_look_for" in their contents and not in their filenames (like Wag's answer).
[131067870020] |From your question it's hard to tell what you are looking for.
[131067880010] |greps parameter -l will only output the filenames which are matching $PATTERN, wc can count them afterwards.
[131067890010] |grep -l "string" * | wc -l will search for "given string" in the contents of all files in the working directory and tell you how many matched.
[131067900010] |Perl find and replace is writing every file
[131067900020] |Here's the test script I'm running
[131067900030] |I need to figure out how to tweak this line:
[131067900040] |To not write every file it finds but to just write the files that need to be changed.
[131067910010] |Try this:
[131067910020] |That searches for the regular expression and returns on the filenames where it is found.
[131067910030] |Then Perl will operate only on those files.
[131067920010] |This should be a suitable replacement:
[131067920020] |grep -l foo * | sed -e 's/[^/0-9A-Z_a-z]/\\&/g' | xargs sed -i 's/foo/bar/g'
[131067930010] |`chmod 770 folderName` restrics access to subdirectories & subfiles ?
[131067930020] |If I run chmod 770 ./folderName, then users who are not the owner or in the group owning ./folderName (i.e. users in the “others” category) cannot access ./folderName/folderB or ./folderName/fileC, even after running:
[131067930030] |Right?
[131067930040] |Does that rule applies to all Linux distributions?
[131067930050] |Thank you.
[131067940010] |That's correct.
[131067940020] |Removing execute permission will prevent access to the directory and all subdirectories, even if subdirectories are more permissive.
[131067940030] |This will probably hold true for anything Unix like (and it may even be a POSIX requirement).
[131067950010] |Year 2038 problem
[131067950020] |What is the likelihood of the Year 2038 issue being very problematic?
[131067960010] |This is my opinion, but this problem is due to a 32 bit counter problem, today most os are updated to handle time on 64 bit (at least on 64 bit computers), so I guess that all OS and software will be ready a long time before 2038, let's say 2010.
[131067960020] |So you might only have problems if in 2038 you will still be running software from 2020.
[131067960030] |It likely not be a problem in almost all case.
[131067960040] |I hope.
[131067970010] |I have encountered this problem in an embedded Linux system that needed to handle dates past 2038 in some long-term cryptographic certificates, so I'd say the likehood of this depends on your application domain.
[131067970020] |While most systems should be ready well before 2038, if you find yourself today calculating dates far into the future, you may have a problem.
[131067980010] |A 64 bit OS is ultimately irrelevant to the 2037 problem.
[131067980020] |(CTIME runs out closer to 2037 than 2038).
[131067980030] |The question is not the bit depth of the OS, rather how does the OS store time.
[131067980040] |Or how does the database column choose to store time.
[131067980050] |Or how does this directory services time syntax attribute store time at the back end.
[131067980060] |This is a much bigger problem than people think, since it is so endemic and common to have used 32 bit time counters.
[131067980070] |Each instance that stores time needs to be revisited, and all API's updated, and all tools that use it updated as well.
[131067980080] |Abstraction layers that let you set time via a human readable time format, instead of the raw data written out make it easier, but that is only one case.
[131067980090] |I suspect this is going to be a much bigger deal than most people think.
[131067990010] |I think this is going to be a significant problem, much more pernicious than the Y2K issues of 1999/2000 because the affected code is generally lower-level (it's CTIME) and so it's harder to spot places where time is being stored that way.
[131067990020] |To complicate matters further, the fact that Y2K was perceived to be a damp squib will make it harder to draw attention to the problem in the runup to the event.
[131067990030] |Cultural references:
[131067990040] |Cory Doctorow was trying out a new model for short story commisioning / publishing under open licenses, and I suggested a 2038 theme o one of them, which he did brilliantly in Epoch: http://craphound.com/?p=2337
[131068000010] |A few years ago, there were reports of problems already, in areas like mortgage programs doing calculations on 30-year loans: 2008 + 30 = 2038.
[131068010010] |The Y2K problem was two charters representing the year instead of four.
[131068010020] |Many system had no way to distinguish between 2000 from 1900 since they only stored the '00'.
[131068010030] |Almost all system either now use 4 chars to store the year, or they use a library of some sort.
[131068010040] |So lets all worry about the year 10000 (Y10K) instead.
[131068010050] |Except for the OS and Library writers...
[131068020010] |it should be 1 instead of 9 but ctime does not handle larger date:
[131068020020] |My system (64 bit of course) time can run even 1 million years more.
[131068020030] |The solution is to update the systems to 64 bits.
[131068020040] |The catch is that the programs may not handle it.
[131068020050] |Especially old, propertary and not maintained.
[131068020060] |Devs are used to following facts:
[131068020070] |int's are 32 bits (in fact they are preserved as 32 bits on 64-bit systems among others because it was assumed they are 32 bit always)
[131068020080] |Most types (such as time_t) can safely be casted on int
[131068020090] |In popular FLOSS software both things will probably not get through the 'many eyes' review.
[131068020100] |On less popular and propertary it will largly depend on author.
[131068020110] |I guess on free *nix world the 2038 will get 'unnoticed' while I do expect problems on "propertary" platforms (i.e. those with large number of propertary software) - especially if some of the crutial part will not be maintained.
[131068030010] |If it's anything like Y2K, some people have already been affected and are changing software, but most developers will ignore it until sometime in the 2030s, probably 2035 or so, at which point there will be a lot of work done, and anybody old enough to know K&R C and not yet too senile will suddenly be able to contract out for a lot of money.
[131068030020] |The actual transition will show a lot of things not yet done, probably none all that important.
[131068040010] |Not such a big deal.
[131068040020] |During the first Y2K blitz, in which software and hardware vendors were required to certify their products as "Y2K compliant" in order to be sold (I remember network cables on PC Connection being certified Y2K compliant) a lot of companies did detailed audits of everything, by setting clocks in the future and testing.
[131068040030] |At the time, since the cost of testing was so high, they almost always tested with several dates, such as 1/1/99 (some developers may have used 99 as a sentinal), 12/31/99, 1/1/00, the leapness of 2000, 1/19/38, and many others.
[131068040040] |See here for a tedious list.
[131068040050] |Thus I believe that any important software that was around in 1999 will probably not have 2038 bugs, but new software written since then by ignorant programmers might.
[131068040060] |After the whole Y2K debacle programmers generally became much more aware of date encoding issues so it's unlikely to be as big an impact as Y2K was (which, in itself, was something of an anticlimax).
[131068050010] |Byobu/screen constantly freezes in Putty
[131068050020] |For a while now I've been using Byobu to manage screen sessions for me.
[131068050030] |However I've run into a strange issue: The session keeps locking.
[131068050040] |By locking I mean that I can't type anything, and only if I wait a few minuites does unlock and do everything I typed in before.
[131068050050] |Currently its getting worse.
[131068050060] |The session keeps locking every minute or so.
[131068050070] |Even when I've restarted Putty, switched from Private Key auth to password auth, and even changed Putty installations, it still freezes.
[131068050080] |Its gotten to the point where I'll login and the current window is frozen but I'm able to switch to another window, only for that to freeze in a few seconds.
[131068050090] |Another new development is that sometimes when I first login Byobu won't even come up, it will just sit there after the "Last login" line.
[131068050100] |One time I got a WriteMessage: Broken pipe error and it dropped to normal shell without byobu.
[131068050110] |Strangely the tunnel's I've opened up still work.
[131068050120] |I was able to check the CPU graphs and saw that the server wasn't under any load, meaning that the session didn't freeze because it had other things to do.
[131068050130] |Any suggestions on how to fix this?
[131068060010] |Show Display Numbers on each Display
[131068060020] |On Windows, there is an option to click on a button in the display properties called like "Identify Displays", it then shows a number over each monitor for a few seconds so that you know which display is which.
[131068060030] |Is there something similar in Linux?
[131068060040] |I am running Arch, and have six (6) identical LCD flat panels, 3 dual head NVidia cards.
[131068060050] |So you can imagine why I ask :P
[131068070010] |Note that (as pointed out by Wieland H., this answer assumes a well-behaving video driver.
[131068070020] |Nvidia's proprietary driver (which used to be required for dual-head operation, I don't know if this is still the case with Nouveau) doesn't use the standard RandR extension, so if you use it most of my answer will be useless and you need to look for an answer that's specific to the nvidia driver.
[131068070030] |Under KDE, the “Multiple Monitors” tab in the “Display” panel in the Control Center (kcontrol, KDE 3) or System Settings (systemsettings, KDE 4) has just this (click on “Identify all displays”).
[131068070040] |You don't have to be running KDE as a desktop environment to use this, you just have to have the right component installed.
[131068070050] |If you prefer to do things by hand, the following snippet parses the output from xrandr.
[131068070060] |It specifically looks for lines like DVI-0 connected 1600x1200+3200+0 …, which gives the position (3200 pixels from the left, 0 pixels from the top), size (width 1600, height 1200), and name (DVI-0) of a monitor.
[131068070070] |It displays a pop-up with this information using kdialog.
[131068070080] |The script kills all kdialog instances as soon as one of them returns, which happens when you click on any pop-up.
[131068070090] |Any other program similar to kdialog would do, as long as you can specify the position of the pop-up (this isn't the case with the better-known zenity).
[131068070100] |The shell has to be ksh (it can be pdksh or ATT ksh), because other shells (at least, bash and zsh) don't behave well enough in the presence of multiple jobs (specifically, their wait built-in isn't interrupted by a SIGCHLD).
[131068080010] |How do I debug a kernel module in which a NULL pointer appears?
[131068080020] |I have a custom kernel module that I compiled from this patch that adds support for the logitech G19 keyboard among other G series devices.
[131068080030] |I compiled it just fine against Ubuntu's maverick kernel's master branch (2.6.35).
[131068080040] |I can boot and load the module, but I'm running into a really strange situation.
[131068080050] |As soon as I load the module (either on boot or through modprobe), I get a black screen and my console locks up.
[131068080060] |The weird part is that it doesn't lock my system up, it's just the current console session.
[131068080070] |I can SSH into my box, and it gives me a terminal and a session.
[131068080080] |And I can type, and I can even run a command and it gives me the output.
[131068080090] |It then draws my next prompt and immediately locks up.
[131068080100] |I see in dmesg that there's a null pointer, and I get the following stacktrace:
[131068080110] |Can anyone point me in the right direction as to how to go about debugging this?
[131068080120] |The stacktrace leads me to believe that it's not the hid-g15 driver but the hid-gfb driver, which creates a frame buffer for the LCD on the keyboard.
[131068080130] |This makes sense since it's locking up my display/console but digging into the kernel code isn't really going anywhere.
[131068080140] |So much of it is assembly and macro functions.
[131068080150] |The last function on the stacktrace that involves my new code is gfb_fb_imageblit.
[131068080160] |The entirety of that function is
[131068080170] |Am I reading the stacktrace wrong?
[131068080180] |Am I missing something?
[131068080190] |Any tips on how to debug this?
[131068090010] |First things first, debug the module?
[131068090020] |Just see if you can load it up in gdb it might point you straight at a line that uses the relevant variable(or close to it).
[131068090030] |oh, and you might find this article useful
[131068100010] |svn switch --relocate: wrong uuid?
[131068100020] |I am trying to switch to a new SVN repository.
[131068100030] |I currently have the following checkout:
[131068100040] |Now I wanted to switch, but I get an error:
[131068100050] |I thought the whole point of svn switch --relocate was that there are two repositories with different uuids?
[131068100060] |How can I correctly switch to the new repos?
[131068100070] |BTW, the directories should be correct: The old proj/trunk and the new trunk/proj contain exactly the same files etc.:
[131068100080] |I am using svn 1.6.5 on the Mac.
[131068110010] |The --relocate flag updates your working copy if the URL used to reach your existing repository changes.
[131068110020] |There's not a good way for SVN to switch a working copy to a new repository, because even though it might contain the same files, any number of other things might be different -- the history, the number of checkins, the contents of the files, et cetera, and SVN isn't designed to account for those sorts of differences.
[131068110030] |You've asked your question purely from the client perspective, but it would be useful to have a bigger picture of what you're trying to do.
[131068110040] |For instance, are you trying to move your own repository to a new server?
[131068110050] |If so, what method did you use, and might there be a better one?
[131068110060] |Or have you been using a repository managed by someone else and they moved it?
[131068110070] |Is all of the history of the old repository present in the new repository?
[131068110080] |Is there a reason you/they can't copy the SVN database files to the new server instead of creating a brand new repository?
[131068110090] |There is a way to force what you want -- namely, delete your .svn directories from your working copy (make a backup first!), check out a new copy from the new server, and move all of the .svn directories from the new copy to the corresponding places in your working copy.
[131068110100] |But only do that once you've considered the bigger picture.
[131068120010] |svn switch --relocate expects to find the same repository at a new URL, not a different one.
[131068120020] |Did you use svnsync to create the new repository?
[131068120030] |In that case you can use svnadmin setuuid to use the same uuid as the old repository.
[131068120040] |According to the svnsync documentation in the svn book, this should enable you to reuse existing working copies.
[131068130010] |Clean up / after accidentally extracting archive into it?
[131068130020] |Possible Duplicate: How to de-unzip, de-tar -xvf —de-unarchive in a messy folder?
[131068130030] |I have just accidentally extracted a cpio archive as root into /.
[131068130040] |Now my / is littered with junks.
[131068130050] |What is a proper way to clean up those files?
[131068140010] |Questions on Linux kernel internals
[131068140020] |Hi
[131068140030] |I was reading "Linux device drivers, 3rd edition" and faced a few kernel items I don't quite understand.
[131068140040] |Hope gurus on this forum will help me out.
[131068140050] |Does the Linux kernel internally operate with virtual or physical addresses?
[131068140060] |What especially confuses me is that there are several types of addresses (logical, virtual, bus and physical) and they are all valid and operable by the kernel.
[131068140070] |Is that correct that CPU instructions can't directly address data stored in peripheral devices and therefore addressable memory is used, i.e. buffers, for these purposes?
[131068140080] |Can a process sleep when requesting a semaphore (which has a value 0) and has to wait for it?
[131068140090] |Atomic operations -- are these guaranteed by specific CPU instructions?
[131068150010] |I can try to answer 1,3 and 4.
[131068150020] |The Linux kernel uses different steps to transition program code into electrical signals.
[131068150030] |Logical Address: Those are included in the machine language instructions to address an operation or instruction.
[131068150040] |Divided into segment and offset.
[131068150050] |Linear Address: The Segmentation Unit translates logical into linear addresses.
[131068150060] |This is a hexadecimal number (on 32bit architecture: 0x00000000-0xffffffff) addressing the space in memory.
[131068150070] |Physical Address: Further, the Paging Unit converts linear into physical addresses.
[131068150080] |Those are electrical signals addressing memory cells over pins on microprocessors.
[131068150090] |Bus Address: Used by all hardware devices except the CPU to address memory cells (DMA does not require the CPU, but still addressing).
[131068150100] |These addresses are mostly identical to the physical ones, except on some other architectures, like SPARC and Alpha which include a separate I/O Memory Management Unit.
[131068150110] |The kernel operates with all addresses, and every of them is a step between the request of a user and the actual processing of this request on hardware level.
[131068150120] |If a process approaches a semaphore with value 0 or lower, he gets suspended until the value reaches 1 or more.
[131068150130] |This only occurs to processes which can sleep.
[131068150140] |An interrupt handler cannot sleep and therefore is forbidden to use semaphores.
[131068150150] |Atomic operations can be achieved by using Assembly language instructions, those are defined by:
[131068150160] |Zero or one memory access
[131068150170] |Prefixed with LOCK_PREFIX
[131068150180] |On C level, the kernel provides the type atomic_t and macros prefixed with atomic_ (which add LOCK_PREFIX to the assembly instructions).
[131068160010] |It's better not to ask multiple questions at once as not everyone can or will answer everything.
[131068160020] |Still, I'll give a short answer to each.
[131068160030] |Does the Linux kernel internally operate with virtual or physical addresses?
[131068160040] |What especially confuses me is that there are several types of addresses (logical, virtual, bus and physical) and they are all valid and operable by the kernel.
[131068160050] |Yes.
[131068160060] |Different parts of the kernel use different address spaces.
[131068160070] |While kernel code is processing a system call, its memory mappings include both the whole kernel memory space and the whole memory space of the process (unless your kernel has high memory configured in, but that's too complicated to go into here).
[131068160080] |These are all logical (a.k.a. virtual) addresses: the high-order bits of the address indicate which page to look up in the MMU, and the low-order bits are a linear address inside the page.
[131068160090] |The memory mappings inside the MMU change whenever a task switch occurs (changing the page table in the MMU is a big part of a task switch).
[131068160100] |Some device drivers need to manipulate memory addresses that are valid for the device they are driving.
[131068160110] |These are often physical addresses, although some architectures have an IOMMU so that devices also see logical addresses of their own.
[131068160120] |Of course, the memory management subsystem in the kernel needs to manipulate lots of different kinds of addresses.
[131068160130] |Is that correct that CPU instructions can't directly address data stored in peripheral devices and therefore addressable memory is used, i.e. buffers, for these purposes?
[131068160140] |This is architecture-dependent.
[131068160150] |Most architectures do have some kind of DMA (direct memory access), that allows at least some communication with devices to be done via RAM.
[131068160160] |Additionally, on some architectures (e.g. ARM) all device accesses are done with load and store instructions at appropriate addresses, while others (e.g. i386) have specific processor instructions for that purpose.
[131068160170] |See Memory-mapped I/O for more details.
[131068160180] |Can a process sleep when requesting a semaphore (which has a value 0) and has to wait for it?
[131068160190] |Yes, taking a semaphore (down and friends) is a blocking operation.
[131068160200] |This is well-explained in the book.
[131068160210] |Atomic operations -- are these guaranteed by specific CPU instructions?
[131068160220] |Yes.
[131068160230] |The details are very architecture-specific.
[131068160240] |All platforms intended for multitasking provide at least one atomic primitive for synchronization such as compare-and-swap, test-and-set, load-link+store-conditional, etc.
[131068160250] |In addition to using the correct primitive, the code may need to take care to use proper memory barriers on multiprocessor systems.
[131068160260] |The Linux kernel provides an implementation of its synchronization primitives for each architecture it supports, so that you in turn only need to use the kernel's portable primitives.
[131068170010] |Prevent non-root user from formatting a partition
[131068170020] |Hello,
[131068170030] |Few days back I mistakenly formatted a partition on my external hard drive by clicking Format from the Context menu in Computer.
[131068170040] |I want to know that how can I prevent non-root user from being able to do so.
[131068170050] |At the same time I need the non-root user to be able to read and write on the partition.
[131068170060] |I use Fedora 14.
[131068170070] |Thanks.
[131068180010] |This is most likely a case of policy elevation.
[131068180020] |When you do it from an honest terminal this is what happens:
[131068180030] |When you do it from gnome or whatever there's a daemon whose task is to elevate your privileges when you want to perform a superuser action (it usually warns it's a privileged operation and asks for a password).
[131068180040] |I believe Fedora uses PolicyKit but these systems work similarly.
[131068180050] |The real danger of most of them is that they offer to "Keep this password in the keyring" or something like that and don't warn you again.
[131068180060] |sudo,[6] gksudo, and kdesu do not ask the user to re-enter their password every time it is called to elevate a program.
[131068180070] |Rather, the user is asked for their password once at the start.
[131068180080] |If the user has not used their administrative privileges for a certain period of time (sudo's default is 5 minutes[6]), the user is once again restricted to standard user privileges until they enter their password again.
[131068180090] |So the solution is to use a user that simply doesn't have the right to perform these tasks, even if he enters his password.
[131068190010] |Fedora has some serious security problems if it allows non-root users to format the disk.
[131068190020] |Best thing to do is this:
[131068190030] |Then, if by needing the "non-root user to be able to read and write on the partition" you mean reading and writing to the file system, then it should be fine provided it is mounted.
[131068190040] |If you mean the other thing, then modifying the permissions on the program that is doing the reading and writing (possibly by running it through sudo) might solve the issue.
[131068190050] |Other than that, I think the partition structure is stored in the superblock of the disk (e.g. /dev/sda -- no numbers) and thus, setting that to 0600, and the partitions to a more permissive mask (like 0660 and group-owned) might solve the problem.
[131068200010] |As nc3b already pointed out, this gets controlled by PolicyKit.
[131068200020] |The policy for disks is located at: /usr/share/polkit-1/actions/org.freedesktop.udisks.policy and can be adjusted.
[131068200030] |Open it with root rights and search for the line: no:
[131068200060] |Good, next time you try to format a device as a non-root user, either over the context menu or over 'Disk Utility', an error message will appear an disallow it.
[131068200070] |This step will still allow the non-root user to read/write the device.
[131068200080] |If you still want to allow formating of devices, but with a higher security level, you can force PolicyKit to ask for a password every time.
[131068200090] |Open the same file and go to the same section, substitute the 'yes' with 'auth_admin' in allow_active:
[131068200100] |Check:
[131068200110] |Excellent!
[131068200120] |Note: I've only tested this on Ubuntu, but Fedora also uses PolicyKit, so try it with a dummy drive first.
[131068210010] |What is the easiest way to boot Linux from a USB drive on Intel Mac?
[131068210020] |I would like to boot an Intel Mac from Linux on an external USB drive.
[131068210030] |Right now, I have install rEFIt and use grub on the internal hard disc to boot the external drive.
[131068210040] |That seems pretty complicated.
[131068210050] |Isn't there an easier way?
[131068210060] |In particular one that does not involved installing anything on the internal drive?
[131068220010] |Using Diskutil
[131068220020] |Download the desired .img file
[131068220030] |Open a Terminal
[131068220040] |Run diskutil list to get the current list of devices
[131068220050] |Insert your flash media
[131068220060] |Run diskutil list again and determine the device node assigned to your flash media (e.g. /dev/disk2)
[131068220070] |Run diskutil unmountDisk /dev/diskN
[131068220080] |Execute sudo dd if=/path/to/downloaded.img of=/dev/diskN bs=1m
[131068220090] |Run diskutil eject /dev/diskN and remove your flash media when the command completes
[131068230010] |Apple doesn't want it to be easy!!!
[131068230020] |You can perhaps ask the rEFIt devs to make the specific process you want a bit easier, but I don't think you can get without having to install something to circumvent Apple-imposed restrictions.
[131068240010] |All you need to do is format the usb stick in a format linux can boot from, for example ext3.
[131068240020] |Then you copy the content of a live-cd or full distro on the USB stick.
[131068240030] |Afterwards, you need to mark the USB parition as 'ACTIVE' somehow.
[131068240040] |Then you need to go into the BIOS of the computer and set the primary boot device to USB.
[131068240050] |Then you restart.
[131068240060] |If all goes well, Linux boots.
[131068240070] |If it doesn't, something was done incorrectly.
[131068240080] |It's clearly possible, I've done it on my Dell notebook.
[131068250010] |Run Python script on startup
[131068250020] |Running on Ubuntu Karmic.
[131068250030] |I need to execute the following (updated) script at boot time
[131068250040] |Have placed it into /etc/init.d Registered it with
[131068250050] |But nothing happens on boot.
[131068250060] |I have tried looking in the logs but I cant find anything.
[131068250070] |Both scripts and liteserv.py are marked as executable.
[131068250080] |As per Andy Smith's response, When I run:
[131068250090] |I now get the program running correctly (after correcting the bad quotes).
[131068250100] |However, it still does not run the program on bootup.
[131068260010] |You can add your stuff to /etc/rc.local.
[131068260020] |Otherwise, you could place your script into /etc/init.d/, chmod +x it and then run update-rc.d FOO defaults.
[131068270010] |Your script indicates it wants to be started in runlevels 3 and 5, stopped in runlevels 1 and 2 (plus 0 and 6 which are shutdown and reboot), and no opinion in runlevel 4.
[131068270020] |Ubuntu boots to runlevel 2 by default (I think this is still true with Upstart).
[131068270030] |So unless you set a different default runlevel your script won't run.
[131068270040] |In the script headers, use
[131068280010] |This is bizarre, but unless it is something that Stack Exchange is doing, I think the problem you are having is that you are using "fancy quotes" rather than "".
[131068280020] |Whatever editor you are using may have replaced the standard quote character(Unicode U+0022) with stylized open and close quotes (U+201C and U+201D). sh doesn't like this very much.
[131068280030] |When I replace these stylized quotes with "normal quotes" your code works fine (with the small exception that I don't have the .py file it calls).
[131068290010] |By adding:
[131068290020] |at the beginning of the scripts file I was able to at least trap the error messages that was coming from the Python program (an authentication error with postgres).
[131068290030] |So the script is running at boot time (after much community input).
[131068290040] |It just fails during the run (which is a different problem).
[131068300010] |How do I suppress dd output?
[131068300020] |I have a bash script that creates some file using dd.
[131068300030] |The problem is dd throws a great amount of output that is going to mess with the output of my script.
[131068300040] |Searching around I've found a solution:
[131068300050] |Is there an alternative, or is redirecting to /dev/null the only way?
[131068310010] |From the dd(1) man page:
[131068310020] |thus:
[131068310030] |This still outputs the
[131068310040] |garbage when dd exits, so redirecting to a data sink really is your only option.
[131068320010] |If I understand correctly what you are trying to do, are you putting that sudo command into the script and expecting the script to prompt for your password when it runs there?
[131068320020] |In that case you are just doing things the complicated way.
[131068320030] |A cleaner solution is to write the script in the usual way (i.e without sudo) and run it as the superuser.
[131068320040] |The reason behind this is, if the script needs superuser access, then just give it the access (why wait until a certain command?).
[131068320050] |In the script, to check if it is being run as root do something like this:
[131068330010] |you can redirect the output to an regular file i.e.:
[131068340010] |Something like this should also work for you with recent versions of BASH and ZSH:
[131068340020] |P.S.
[131068340030] |This is just an example I ran...
[131068350010] |With any Unix application or command, you can suppress all output with
[131068350020] |cmd >/dev/null 2>&1
[131068350030] |The first bit redirects the standard output (unit number 1) to /dev/null.
[131068350040] |But you need the second part to ALSO redirect the error output (unit number 2) to the same place as number 1.
[131068350050] |In UNIX, STDIN=0, STDOUT=1 and STDERR=3
[131068360010] |Problem with Fedora 14 after ATI Catalyst 10.11 Install
[131068360020] |I followed the steps as given in this article.
[131068360030] |After following all of the steps, ATI Catalyst was installed but when i boot my system, I get these options at GRUB:
[131068360040] |By choosing the second one, a lot of commands appear on screen and after that the screen goes blank but the system keeps running
[131068360050] |Whenever I start up my system, fedora boots a little late
[131068360060] |The command line interface for fedora appears in very large font
[131068360070] |Even the graphics appear poor
[131068360080] |System information:
[131068360090] |Fedora Release 14(laughlin)
[131068360100] |Kernel linux 2.6.35.10-74.fc14.i686
[131068360110] |Gnome 2.32.0
[131068360120] |ATI mobility radeon HD 4500
[131068360130] |Video Processor ATI Radeon Graphics Processor (0x9553)
[131068360140] |Driver Version 8.553.2.0
[131068360150] |Is there any way to correct such behavior?
[131068360160] |If not, please tell how to remove the ATI Catalyst.
[131068370010] |You usually want to install Catalyst from the RPMFusion.org repository.
[131068370020] |It does a better job of keeping the kernel module updated anytime your kernel gets updated.
[131068370030] |But the installer from amd.com should work for the time being.
[131068370040] |The second option is an older kernel.
[131068370050] |When you installed Catalyst it only built a module for the kernel you were running at the time of install which would have been the first option.
[131068370060] |If your kernel doesn't have the Catalyst module, you get no graphics at all.
[131068370070] |I'm not sure what you mean by late?
[131068370080] |Do you mean it takes longer to boot?
[131068370090] |Yes, the console font should be larger.
[131068370100] |Before you installed Catalyst you were using the open source Radeon drivers which implements something called KMS (Kernel Mode Setting) which gives you a high resolution console.
[131068370110] |Since KMS is fairly new and heavily reliant on the kernel and a lot of open source technologies, the closed source Catalyst driver doesn't use it.
[131068370120] |Maybe it will when that stuff gets more stable.
[131068370130] |Not sure what you mean by the graphics being poor.
[131068370140] |Did you set the resolution of your monitor to be the max it can support?
[131068370150] |If you're running at 800x600 then yeah it would look poor.
[131068380010] |FreeBSD's kernel is 'locked'?
[131068380020] |I happened to talk to a Mac Genius on the subject of viruses.
[131068380030] |He mentioned that the FreeBSD kernel on which the Mac OS is based is 'locked' and therefore no viruses can ever harm my Mac.
[131068380040] |I didn't quite understand what he meant.
[131068380050] |Could someone explain?
[131068390010] |He didn't know what he was talking about.
[131068390020] |There are a couple things I can think that he's talking about:
[131068390030] |The user logs in as a non-superuser, so user processes can't simply load modules or make changes to the kernel in other ways. (of course, any macosx viruses would exploit flaws in the kernel to bypass that, or just use social engineering to get the user to type in the password)
[131068390040] |MacOSX has better memory protection than versions earlier than 10.0.
[131068390050] |Perhaps the "genius" is remembering the days with MacOS 9 and previous?
[131068390060] |Nearly every modern OS has that now, and it hasn't stopped virus writers.
[131068390070] |I think this Mac Genius probably was just mis-remembering some marketing.
[131068390080] |MacOSX can get viruses.
[131068390090] |There are certainly some advantages to using a Unix-like system, and perhaps the BSD Userland + Mach microkernel does also provide some added protection, but it is demonstratively false to say that the MacOSX kernel is somehow protected from viruses.
[131068400010] |sudo permission denied but su grants permission
[131068400020] |Hi all, this is the first occurrence where su was required for me.
[131068400030] |I read an article about changing the value in /sys/devices/virtual/backlight/acpi_video0/brightness to alter my laptop's screen brightness.
[131068400040] |I first noticed that when I would $ sudo echo 10 >brightness I would get permission denied.
[131068400050] |I switched to root using su and # echo 10 >brightness changed my brightness almost instantly.
[131068400060] |The last weird thing to me happened when I tried # echo 20 >brightness (maxbrightness file holds the value 15) and I got a write error
[131068400070] |Could someone explain this difference between sudo and su to me?
[131068400080] |Understanding the write error would be an added bonus.
[131068400090] |Any help, pointers, and/or links would be much appreciated.
[131068400100] |Thanks
[131068410010] |Redirection does not work that way.
[131068410020] |Appending >to a command will run that redirection as the invoking user (you) and not as root.
[131068410030] |Do it with tee:
[131068410040] |or by invoking the command in a separate privileged shell:
[131068420010] |This isn't because of sudo, it's because of the way your command is processed; I actually explained it in this question.
[131068420020] |When you do
[131068420030] |the shell runs the command sudo echo 10, which runs echo 10 as root.
[131068420040] |The shell then tries to open brightness so it can redirect the output from echo 10 into it, but it can't -- your shell is running with your permissions, not root.
[131068420050] |There are workarounds posted as answers on the question I linked to; a good one is:
[131068420060] |Now tee is the one to open brightness, and since it's running as root it succeeds
[131068430010] |You might want to take a look at /etc/sudoers file.
[131068430020] |If your a/c is not listed there then you will not be able to sudo.
[131068440010] |How do I check what 'normal' users are there on the system?
[131068440020] |I want to find out what 'normal' users are available on a system.
[131068440030] |By normal, I mean those who are manually created, not those like sshd or gdm.
[131068450010] |I don't think there's a way to tell which users have been "manually created".
[131068450020] |Probably the best you can do is look for users that have a valid shell, as typically system users like sshd and gdm have shells like /bin/false and /sbin/nologin to prevent logins.
[131068450030] |The list of valid shells is in /etc/shells, so you can use grep to get a list of users with one of those shells:
[131068450040] |If you just want usernames:
[131068460010] |But that's also not the most reliable source.
[131068460020] |If once logged in:
[131068470010] |Typically "normal users" are assigned user ID's >= 1000.
[131068470020] |If this is the case on your system, then the following could work:
[131068480010] |Do grep UID_MIN /etc/login.defs.
[131068480020] |Then do getent passwd and look which users have a UID >= the UID_MIN value from above.
[131068480030] |Explanation: Whenever a user is manually added, it grabs the first available UID >= the UID_MIN value from login.defs (unless UID is manually specified)
[131068490010] |Am getting a blank screen during installation stage of Arch Linux
[131068490020] |I used dd if=archlinux-2010.05-core-x86_64.iso of=/dev/sdb to create an installation image on a USB stick and I'm getting a blank screen soon after boot-up.
[131068490030] |The last message I saw is:
[131068490040] |What am I to do?
[131068490050] |[note] Am using a TravelMate 6592: Core 2 Duo T7500 2.2GHz processor.
[131068500010] |Try adding nomodeset and a low resolution like vga=771 to the kernel line in GRUB.
[131068510010] |What Linux utility can split a large file into a multi-file archive that can be opened on Windows?
[131068510020] |I have a 6GB file which I need to transport on DVDs.
[131068510030] |I have a Linux system, while the recipient has a Windows system.
[131068510040] |Is there any way for me to make a multi-part archive to spread the file across two DVDs that can be opened by a Windows user?
[131068520010] |Rar is the common utility for operations such as this as it is available on both platforms.
[131068520020] |However another approach is to use split.
[131068520030] |split -b 1024m -d somefile somefile.
[131068520040] |Then use 7zip on the windows machine to re-combine the files.
[131068520050] |Rar has the advantage of being a little simpler, but requires it to be installed on both machines.
[131068520060] |Split has the advantage of it being faster as it doesnt have to put the data in an archive, but leaves it as is.
[131068530010] |Make a zip archive and split it with
[131068530020] |I believe recent versions of Windows have built-in support for zip archives.
[131068530030] |If the recipient's system is too old, it may have Winzip or 7-zip.
[131068530040] |If you have a single file, you can split it with split:
[131068530050] |On the Windows machine, on the command line (press Win+R and type cmd), assemble it with
[131068540010] |How do I ensure that a terminal remains scrollable?
[131068540020] |On my tty terminals (1-6), I lose text when I move away from it and back (i.e. pressing ShiftPg Up doesn't reveal any scrolled-off text).
[131068540030] |How can I recover this text?
[131068550010] |As far as I know, it's just lost.
[131068550020] |The Linux console only stores history until you change to a different console.
[131068550030] |If you want to keep more history, use Screen.
[131068560010] |Why is the temporary system needed to build a Linux from Scratch ?
[131068560020] |Hi
[131068560030] |I just quickly read the Linux From Scratch book and found it very interesting.
[131068560040] |However I don't understand the role of the temporary system (Chapter 5).
[131068560050] |It is built from a working distribution, and then we use it to build the final system.
[131068560060] |Couldn't we just make a partition and build a system on it ?
[131068570010] |You could use a partition on an existing Linux system.
[131068570020] |That would be the same as using the temporary system.
[131068570030] |However, assuming you are using an empty system, you will need something to build the LFS software with.
[131068570040] |You can't just drop a compiler on a partition and start churning away.
[131068570050] |You need a little more than that.
[131068570060] |A kernel, for example, and a boat load of libraries :)
[131068580010] |I don't have a copy of that book, but I can explain why a two-step process might be desirable if you're building a complete distro.
[131068580020] |When building packages, it sometimes happens that properties of the build system creep into the binaries that you build.
[131068580030] |There are library paths, configuration variables etc, etc.
[131068580040] |At Sun/Oracle the standard process for building Solaris is that you need to build biweekly build 17 on a system that's running biweekly 16 (for example).
[131068580050] |For example, there are many fields and structures in header files, you need to make sure that you're building the binaries using the header files that will be delivered on the system.
[131068580060] |But some of the packages you are building deliver their own header files.
[131068580070] |And those headers are used by other packages.
[131068580080] |Hopefully I've given you an idea of how the system you are building can subtly depend on the system used to build it.
[131068580090] |So you would usually want to build on a system that is as close to the system you're building as possible.
[131068590010] |I'm moving up from managed hosting to a VPS, what do I need to know?
[131068590020] |I'll be starting to use a VPS for my web hosting this week, which is a big step-up for me since I've only used managed/shared services before.
[131068590030] |I've a bit of experience with Linux before and I'm not afraid of the terminal but this is a completely different environment from the one I'm used to operating in.
[131068590040] |I'm mainly concerned about security, while there's not going to be anything critical there for a while I do intend to make this a learning exercise.
[131068590050] |From what I know, I want to:
[131068590060] |Configure user accounts with appropriate permissions,
[131068590070] |Only allow certain ports to be opened (HTTP, HTTPS, FTP and/or FTPS and SSH),
[131068590080] |Pay attention to directory/file permissions in the webroot of applications for the HTTP user
[131068590090] |I haven't done much SSH/VNC so I intend to read up on that too.
[131068590100] |What advice could you offer me in my case?
[131068590110] |While I'd like this to be fairly generic I'll be using Ubuntu Server 10.04 (although I have the choice between CentOS and Debian (or Windows) as alternatives) and Apache for the serving websites.
[131068600010] |Hi there Ross,
[131068600020] |Assuming that you choose to install Linux on your VPS, you might want to look into iptables for security.
[131068600030] |Just tailor it to the specific distro as the link I gave is for Debian/Ubuntu.
[131068600040] |Installing other services would be fairly easy.
[131068600050] |Good Luck!
[131068600060] |Ismael Casimpan :)
[131068610010] |Definitely review security with a good Linux Security Checklist as the first order of business.
[131068610020] |SANS publishes a nice one.
[131068610030] |Second step is to get rid of packages that you won't use.
[131068610040] |Third step is df -h and record how much disk space the install uses.
[131068610050] |And then fourth step is to tar up the entire disk using the --exclude option to exclude the output tarball.
[131068610060] |For instance: tar --exclude=all.tar.gz cvf - / | gzip -9 >all.tar.gz
[131068610070] |Then download it and use it to make a clone system, more or less, that runs on Virtualbox or whatever VM environment you use.
[131068610080] |From this point on, test changes on the VM before applying to your VPS and take regular data backups.
[131068610090] |Don't change anything on the VM or the VPS without writing it down on paper in a logbook.
[131068610100] |Even the smallest change should be recorded.
[131068620010] |Double and triple check your firewall configuration.
[131068620020] |There are plenty of resources online for researching that.
[131068620030] |Next, move SSH to a non standard port.
[131068620040] |If your box is secure, this won't matter as much but you'll have less log spam from script kiddies.
[131068620050] |If there are security issue (able to log in a root, week passwords, etc) moving to a non standard port will reduce the likelihood a script kiddie gets finds your ssh server. but again, this is only security through obscurity.
[131068620060] |So make sure you disable root login and have strong passwords or disable password login all together and use keys based authentication. if you need to leave ssh on 22, then install fail2ban or denyhosts to block brute force login attempts.
[131068620070] |Did I mention strong passwords?
[131068620080] |Your most likely point of entry or compromise won't be through the OS or the services it runs, rather the software you present to the public.
[131068620090] |So your next priority is to stay current on updates to software such as wordpress, drupal, joomla and other third party applications.
[131068620100] |Also keep on top of security updates to your OS and Apache or whatever is powering your sites.
[131068620110] |Lastly throw in some precautions for catching anything that does get by.
[131068620120] |I'd suggest something like 'aide' which basically checksums your configuration files, take a peak at log files occasionally and watch or any weird spikes in activity.
[131068620130] |Do these steps and read up a bit on a few sites (the pdf linked in the above comment would be a good start) it doesn't take much to be way more secure than any shared hosting platform.
[131068620140] |and if you really want to be secure, try to hack your own server sometime :)
[131068630010] |Scrollability of text when using tmux?
[131068630020] |Scrollability of text in some terminals:
[131068630030] |I notice in this answer that scrolling doesn't work when using tmux.
[131068630040] |But I can scroll in urxvt.
[131068630050] |How can I have that behaviour in other terminals?
[131068640010] |Try setting the alternate-screen setting to off.
[131068650010] |Add set -g terminal-overrides "xterm*:smcup@:rmcup@" to your tmux config and restart tmux.
[131068650020] |If you are not using a terminal matching 'xterm*' you'll have to change it of course.
[131068660010] |Unable to login as normal user on CentOS
[131068660020] |My normal user account isn't working correctly in CentOS, but logging in as root works fine.
[131068660030] |What should I do to debug/fix the problem?
[131068670010] |Check the logs to see if there are any messages of note there.
[131068670020] |Try resetting your password, not saying you don't know it, but a reset will fix a couple possible issues.
[131068670030] |Then check your /etc/passwd file look for the line with your username on it and see if there is anything out of the ordinary i.e. your shell being set to 'nologin' or 'false'.
[131068670040] |Last you could remove the user acct and then add it back.
[131068680010] |Is it possible to use libc at the bootloader stage itself?
[131068680020] |I would like to use some the functions given by libc at the boot loader stage itself.
[131068680030] |Is it possible to get them at that stage of loading?
[131068690010] |Yes.
[131068690020] |The easiest way would be to copy the code you need to the boot loader.
[131068690030] |Which functions are these?
[131068690040] |You should understand that you can't use them if they rely on system calls to fulfill their duty.
[131068700010] |Just use static linking.
[131068700020] |Dynamic linking will not work at that point.
[131068710010] |You'll most likely have to write your own versions of the functions you want, though in some cases you may be able to use libc source code as a starting point.
[131068710020] |The functions in libc itself are written under all the assumptions of a UNIX userspace program, including:
[131068710030] |the presence of the kernel (or more specifically, the syscall interface to the kernel)
[131068710040] |a flat memory model
[131068710050] |a dynamic linking infrastructure (unless statically linked)
[131068710060] |and at the bootloader stage, you have none of these.
[131068710070] |Instead, (by default, under Intel) you've got the BIOS, a segmented memory model, no memory protection, and full reign of the machine.
[131068710080] |It's the same reason you see the custom printk() function in kernel code instead of printf() -- the assumptions that libc's printf() makes just don't apply in kernel space.
[131068720010] |How can I check if a gzipped file is empty?
[131068720020] |Is there a quick way to check if a gzipped file is empty, or do I have to unzip it first?
[131068720030] |example:
[131068730010] |Or with if:
[131068730020] |zcat is sometimes linked to gunzip -c or gzip -cd, if you want to use it as the shorter "form".
[131068740010] |If by 'empty' you mean that the uncompressed file is 0 bytes, you could use gzip --list foo.gz to determine the size of the uncompressed file, it would require some parsing to automate it.
[131068740020] |It looks something like this:
[131068750010] |gzip -l foo.gz | awk 'NR==2 {print $2}' prints the size of the uncompressed data.
[131068750020] |Alternatively you can start uncompressing the data.
[131068760010] |blank screen after kernel upgrade
[131068760020] |I am running ubuntu 10.10 with catalyst ati drivers which I installed from the amd website.
[131068760030] |Recently I took some updates which included a new kernel.
[131068760040] |I cannot boot into ubuntu.
[131068760050] |The screen shows ubuntu 10.10 thing but after that the it goes blank.
[131068760060] |I am using suse grub loader.
[131068770010] |Most probably the new kernel doesn't have the modules that comes with your proprietary driver and fails to use your configuration files.
[131068770020] |If you still have the boot menu that comes with a default Ubuntu installation then you can choose "Recovery mode".
[131068770030] |In the next menu choose to boot into safe mode, and if Ubuntu asks if you want to create a new configuration file just allow it to do so.
[131068770040] |There should be other ways to generate new configuration (like deleting some dot files) but I just know this way.
[131068770050] |Then when everything is OK reinstall the graphic driver.
[131068770060] |Edit: to boot into recovery mode see the question How do I prevent GRUB from using/starting any graphical UI?
[131068780010] |You'll probably have to reinstall your drivers since the drivers rely on the kernel headers.
[131068780020] |Reboot into recovery mode and then blacklist the fglrx module.
[131068780030] |Then you can reboot into your console and reinstall the drivers.
[131068780040] |Fglrx drivers have always been a pain since it makes it ridiculous to upgrade your kernel.
[131068790010] |Is there is a PPA-service equivalent in the Fedora world?
[131068790020] |For Ubuntu there is this convenient PPA service available where you can get your own package repository.
[131068790030] |You can put there source packages which are under an open source license, e.g. development snapshots of your software.
[131068790040] |(The server takes care of creating the right binary package and signs it with the repository key.)
[131068790050] |Users then only need to add the repository address with one command and can install the packages via apt-get.
[131068790060] |Since I am looking into Fedora - is there is something like this available for Fedora packages?
[131068790070] |If yes, what architectures/distribution versions are supported?
[131068790080] |How much space is available?
[131068790090] |What is the way to publish packages for multiple rpm distributions via such a service (e.g. current Fedora version and RHEL)?
[131068800010] |The closest thing I can think of is Koji for Fedora package maintainers.
[131068800020] |If you want to host your own packages, it's pretty trivial to build your own packages locally and put them someplace web accessible.
[131068800030] |You can create the yum repository with the createrepo tool.
[131068800040] |Then you can just create a repo configuration in /etc/yum.repos.d/ that points to that web-based repository.
[131068810010] |OpenSuse Build Service perhaps?
[131068810020] |Don't let the name OpenSuse fool you, it supports other distros as well.
[131068810030] |I don't believe it gives you a yum repository, but it does give you a place for users to download your .rpm files.
[131068810040] |But the coolest thing it does is it will compile your package for you in whatever architectures and whatever versions of Fedora/Suse/etc you want to support.
[131068810050] |Main page: https://build.opensuse.org/ Wiki page: http://en.opensuse.org//openSUSE:Build_Service
[131068820010] |For Fedora their is: http://repos.fedorapeople.org/
[131068830010] |vim on FreeBSD does not react on arrow keys correctly and other vi-like behaviours
[131068830020] |I tried to use vim on FreeBSD (via ssh on Linux xterm-compatible terminal).
[131068830030] |However vim behaves similar to vi out-of-box.
[131068830040] |For example it does not react on delete key, insert F character instead of going up while on insert mode etc.
[131068830050] |On FreeBSD X11 is not installed.
[131068830060] |Edit: As asked I post vim --version
[131068840010] |Are you typing "vim", or assuming that typing "vi" runs Vim?
[131068840020] |On FreeBSD, the command "vi" is likely to get you nvi.
[131068850010] |my guess is following could be the possible reason for this issue:
[131068850020] |May be in your vimrc file by mistake someone has set vim to behave like vi. set compatible
[131068850030] |Your vim is pointing to original vi binary.
[131068860010] |This is generally a terminal setting problem.
[131068860020] |Check your $TERM environment variable on the FreeBSD side, and check what your local terminal setting is.
[131068860030] |This is almost certainly the problem with the arrow keys.
[131068860040] |In addition to this, make sure you have the standard set of "unbreak my vim" .vimrc settings applied.
[131068860050] |If you're sure the terminal settings are correct, but you're still having backspace issues, take a look at vim's help
[131068870010] |How to find files not of a certain type?
[131068870020] |I'm trying to find out all of the different types of files in my unorganized music folder.
[131068870030] |I've been trying this command (to list files of types besides the ones I know are in there):
[131068870040] |But it's not working.
[131068870050] |How would I get that command work?
[131068870060] |Is there a way to do it using -regex?
[131068870070] |Thanks!
[131068880010] |You wrote (redundant parentheses added for clarity):
[131068880020] |Either use -and -not throughout, or use or throughout and finish with -print (meaning: do nothing for this, otherwise do nothing for that, etc, otherwise print).
[131068880030] |You'll also want to limit the search to regular files, otherwise directories will be listed.
[131068890010] |Repackaging RPMs
[131068890020] |We're using HP DataProtector for our backup environment.
[131068890030] |The installation method leaves something to be desired, and we're attempting to automate it in such a way that it makes our Unix admins cringe less often.
[131068890040] |We're a SLES/OpenSUSE shop, so we're attempting to make up a YUM repository with the DP patches.
[131068890050] |I can make the repo just fine, it's just that the patch RPMs aren't configured right.
[131068890060] |The 'Revision' field in the RPM is not set correctly, they're all "1" even though the master RPM I pulled them out of is correctly incrementing.
[131068890070] |I would really like to be able to rebuild these RPMs with the correct Revision, as that would allow the normal update process to deal with these patches instead of the strange way HP wants to handle these.
[131068890080] |The strange way HP wants to handle these requires:
[131068890090] |Setting up an Installation Server with all of the software.
[131068890100] |No problem.
[131068890110] |Allowing root to ssh into client stations to install software that affects xinitd config
[131068890120] |Which in turn requires a passwordless SSH public-key to be placed on all target machines so the install process can remote in w/o prompting.
[131068890130] |Before any deployments can be made, each client must be manually SSHed to by root on the repo server in order to populate known_hosts
[131068890140] |Since we don't allow root logins via SSH, every time we get a patch we have to touch each server's sshd_config to allow them temporarily.
[131068890150] |We've also proven that after the initial install, subsequent patches can just be installed via rpm just peachy.
[131068890160] |So, we'd like to get that into a YUM repo if at all possible.
[131068900010] |Rather than re-package the existing RPM, inspired by HP I packaged it in an additional RPM.
[131068900020] |The new RPM is very simple in that it just has the single patch-RPM inside it, and invokes the rpm command to install it.
[131068910010] |How to clean up file extensions?
[131068910020] |I have a directories with .MP3 files which I'd like to change the extensions to .mp3.
[131068910030] |What's the easiest way to do this?
[131068910040] |I'm think something along the lines of
[131068910050] |Though I know that isn't quite right :) because the substitution isn't correct and I'm not sure how to use both a mv and a sed command with -exec in find.
[131068910060] |Would I need a bash script for this?
[131068910070] |Thanks!
[131068920010] |Here is one way to do this in Bash:
[131068920020] |I've used echo here so the command itself doesn't do anything but print pairs of files names.
[131068920030] |If that list represents the changes you want to make, then you can change echo to something like mv -i -- which will then move your files (and prompt you before overwriting).
[131068920040] |Brief Explanation
[131068920050] |The for iterates through every file matched by *.
[131068920060] |Then, we determine if the extension is already lowercase, if it is we move on, if it isn't, we proceed to move it (or echo it, as the case may be).
[131068920070] |This uses Bash's built in string operations which you can read about here: http://tldp.org/LDP/abs/html/string-manipulation.html
[131068930010] |Use perl-rename, swiss army knife of file-renaming with regular expressions (on ubuntu, this is bundled with the perl package as prename):
[131068930020] |If you're of the careful sort, use the -n and -v options to perl rename to see what it'll actually do.
[131068940010] |In zsh:
[131068950010] |What does *~ mean?
[131068950020] |At the end of a makefile I saw
[131068950030] |I understand the *.class, but what's *~?
[131068960010] |It's basically removing backup files.
[131068960020] |*~ means all files ending in ~.
[131068960030] |Many Unix/Linux systems programs create backup files that end in ~.
[131068960040] |For example, the emacs and nano editors automatically save a backup copy of each file you edit.
[131068960050] |When it saves a file, the old version gets saved using the file name with a tilde (~) added to the end.
[131068960060] |Vim will do the same if you put :set backup in your .vimrc.
[131068960070] |*~ on Unix/Linux is like *.bak on Windows.
[131068970010] |How to use yum to install from a DVD
[131068970020] |I use Fedora 14 and I am a newbie to Linux.
[131068970030] |Some days ago, I used the command yum remove openoffice.org-* and removed openoffice successfully.
[131068970040] |The, I wanted to get it back.
[131068970050] |I have all the packages and dependancies required to install in an installation dvd.
[131068970060] |The problem is that when I try to manually install each package, yum does not check dependencies in the DVD.
[131068970070] |How can I configure yum to look for all dependancies and packages required, in the DVD?
[131068980010] |Sounds like you need to update your /etc/yum.repos.d to include your dvd.
[131068980020] |See here for details: Howto Setup yum repositories to update or install package from ISO CDROM Image
[131068990010] |Probably the easiest thing to do here, to just fix this problem, is to install the "yum-plugin-tmprepo" plugin and do:
[131068990020] |...where you can get the transaction ID from yum history list openoffice.
[131069000010] |Why do some symbolic links affect program behavior?
[131069000020] |One thing that has been puzzling me for some time is this:
[131069000030] |However, executing sudo halt does, of course, not reboot the system.
[131069000040] |Why is that?
[131069000050] |There are several other programs working that way, for example pdflatex.
[131069010010] |Every program can see the full command line that was used to run it (except for wildcards and variables, which the shell expands).
[131069010020] |In a C program, the command line is stored in argv, which is short for argument vector.
[131069010030] |The progam's name is the first element of argv, i.e. argv[0].
[131069010040] |Clearly in the case of halt and reboot, the program is changing its behavior based on argv[0].
[131069010050] |From bash, you can see the full command line used to run a program using ps -p -o cmd or cat /proc//cmdline.
[131069010060] |Note that there is another type of link called a hard link that will have the same effect.
[131069010070] |On my system for example, sudo and sudoedit are the same file with two different names, and different behaviors.
[131069010080] |ls -i can help you find those commands, e.g.:
[131069010090] |See man ln for more details about hard links if you're not familiar with them.
[131069020010] |executing a sh script from the cron
[131069020020] |I have a test.sh script
[131069020030] |when I execute the file as root from command line it works.
[131069020040] |when I set it to crontab -e (is the root cron), is not working
[131069020050] |What do I do wrong?
[131069020060] |Thanks
[131069030010] |According to the man:
[131069030020] |The cron daemon starts a subshell from your HOME directory.
[131069030030] |If you schedule a command to run when you are not logged in and you want commands in your .profile file to run, the command must explicitly read your .profile file.
[131069030040] |The cron daemon supplies a default environment for every shell, defining HOME, LOGNAME, SHELL (=/usr/bin/sh), and PATH (=/usr/bin).
[131069030050] |So cron daemon doesn't know where php is and you should specify the full php path by hand, for example (I don't know your real PHP path):
[131069030060] |Another way is to source the /etc/profile (or your .profile/.bashrc), for example
[131069030070] |This is useful if your .bashrc set the environment variables that you need (i.e. PATH)
[131069030080] |EDIT
[131069030090] |An interesting reading is "Newbie: Intro to cron", don't undervalue the article from the title (It's a reading for everybody), in fact it's well written complete and answer perfectly to your question:
[131069030100] |...
[131069030110] |PATH contains the directories which will be in the search path for cron e.g if you've got a program 'foo' in the directory /usr/cog/bin, it might be worth adding /usr/cog/bin to the path, as it will stop you having to use the full path to 'foo' every time you want to call it. ...
[131069040010] |The cron daemon usually executes your command in a shell where the PATH environment variable is restricted to some system default, e.g. /usr/bin:/bin.
[131069040020] |Probably, your php command is not available in /usr/bin or /bin and thus the script fails when executed via cron and runs successful when not.
[131069040030] |Cron usually reports errors or job messages via a mail to the root user (i.e. when a command returns an exit status != 0 or produces output to stdout/stderr) after the job is finished.
[131069040040] |Depending on your system you have to setup local mail delivery to get these messages.
[131069050010] |This is not related directly to your question, but it matters cron.
[131069050020] |This is a nice web service for creating a crontab file:
[131069050030] |Cron web editor.
[131069060010] |There are four common causes for commands working when typed in a terminal but not from cron, in order of commonness:
[131069060020] |Cron provides a limited environment, e.g., a minimal $PATH, and other expected variables missing.
[131069060030] |Cron invokes /bin/sh by default, whereas you may be using some other shell interactively.
[131069060040] |Cron treats the % character specially (it is turned into a newline in the command).
[131069060050] |Cron doesn't provide a terminal or graphical environment.
[131069060060] |If your job produces any output, including error messages, cron sends you an email with the whole output.
[131069060070] |Make sure you read the mail you receive locally or forward it to an address you read.
[131069060080] |To forward mail from a local account to some other address, put the other address in ~/.forward.
[131069060090] |If the cron job is running as a system user (root, webmaster, …), make sure that user's mail is redirected to you (and any other admin); with most mail setups, put lines like root: elzo in /etc/aliases.
[131069070010] |How can I tell what version of apache I'm running?
[131069070020] |I know about the phpinfo() way but is there any other way?
[131069070030] |I'm using CentOS and I can't find the httpd executable to run httpd -v.
[131069080010] |Either rpm -q httpd or /usr/sbin/httpd -v should work.
[131069090010] |If there's no /dev/md* and not /proc/mdstat, is it safe to use a second disk as a second disk?
[131069090020] |There's two HDs in this server.
[131069090030] |I don't see an md device (see title) or an mdstat file.
[131069090040] |So, can I be safe that the second disk is currently unused?
[131069090050] |Yes, I do want/need the second disk as space much more than RAID-"safety", in case you're wondering.
[131069090060] |OS is Debian sid (aka unstable).
[131069100010] |Well if you want to check if the drive is being used as an md raid device (and just not currently active), the only safe way to do it is to run mdadm -E /dev/sdX
[131069100020] |However if you dont care about whats on the drive (if youre going to wipe it anyway), then it doesnt matter either way.
[131069110010] |Changing the PS1 on an interactive bash subshell easily
[131069110020] |I have a situation where I need to provide a subshell to a user mid-way through a longish process.
[131069110030] |I would like to change the prompt to remind the user that they are in a special subshell and haven't gone through the rest of the process yet.
[131069110040] |I thought that this would do what I want...
[131069110050] |But when I enter that line, this is the output I get
[131069110060] |Is there a simple way around this?
[131069110070] |I could writeup my own custom bashrc... but I'd prefer to preserve the user's usual bash-shell setup.
[131069120010] |You can use process substitution to essentially make a ~/.bashrc that only exists for the bash -i invocation like so:
[131069130010] |Agree with SiegeX (who beat me to it :))
[131069130020] |To explain the behaviour:
[131069130030] |Bash will only start in interactive mode if its std in and out are normal terminals.
[131069130040] |That's not the case when piping in that initial command.
[131069130050] |You can use the -c option to provide commands, but that also precludes interactive mode.
[131069130060] |If you really want to preserve user rc, simply add some test/source statements in the rcfile string to source ~/.bashrc and /etc/bashrc if the files exist.
[131069140010] |I think it would be better to use env PS1="foo: " /bin/bash --norc -i instead of using the --rcfile option.
[131069140020] |The environment is preserved, not overwritten by the dot files and the PS1 environment variable is set before entering the shell.
[131069140030] |You might also want to think about opening a restricted shell, if the calling program gives extra privs. env PS1="foo: " PATH=$RESTRICTED_PATH /bin/rbash --norc -i for whatever value you want for $RESTRICTED_PATH.
[131069140040] |Or have an already prefabricated bashrc file:
[131069140050] |And then call with /bin/bash -rcfile ~/.bashrc.appsubshell -i
[131069150010] |How do I enable a new repository in CentOS?
[131069150020] |Hello, I need to upgrade my version of subversion however, simply running yum update subversion tells me:
[131069150030] |Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile * addons: mirror.nic.uoregon.edu * base: mirror.ubiquityservers.com * extras: centos.promopeddler.com * updates: mirror.chpc.utah.edu Setting up Update Process No Packages marked for Update
[131069150040] |So, I need to know how I can find a repository for subversion-1.6 and the steps involved to impliment this repository on my machine.
[131069150050] |I see this rpm here: http://ftp.redhat.com/pub/redhat/linux/enterprise/5Client/en/os/SRPMS/subversion-1.6.11-7.el5.src.rpm but I don't know that this really helps.
[131069150060] |Thank you very much!
[131069160010] |If you want to install that specific package you can just do
[131069160020] |wget http://ftp.redhat.com/pub/redhat/linux/enterprise/5Client/en/os/SRPMS/subversion-1.6.11-7.el5.src.rpm
[131069160030] |rpmbuild --rebuild subversion-1.6.11-7.el5.src.rpm
[131069160040] |However the rpmforge repo has 1.6.6 in it, so you can enable that repo if you wish.
[131069160050] |rpm -ivh http://packages.sw.be/rpmforge-release/rpmforge-release-0.5.2-2.el5.rf.x86_64.rpm
[131069160060] |yum update subversion
[131069160070] |Assuming of course that x86_64 is your arch, if not you can change that to i386
[131069170010] |pwd without symlinks
[131069170020] |If I do pwd I notice it uses whatever symlinks I used to get into the current directory.
[131069170030] |Can I get it to tell me the "real" directory I'm in ... i.e. the path from the root to my current directory without the use of any symlinks?
[131069180010] |According to the POSIX manpage for pwd, the -P option may be of use:
[131069180020] |-P The absolute pathname written shall not contain filenames that, in the context of the pathname, refer to files of type symbolic link.
[131069180030] |Thus
[131069180040] |should be what you need.
[131069190010] |The pwd shell built-in uses the path the shell keeps track of when you cd (and stores it in $PWD).
[131069190020] |This means if you have a symlink to a complex (deep) path, it will tell you what you typed to change to that directory instead of the real path.
[131069190030] |This is done to give you what you want most of the time.
[131069190040] |/bin/pwd uses the getcwd system call (which these days is a library call, reading /proc/self/cwd) which returns the canonical path for the current directory, sans all symlink traversals.
[131069190050] |As Steven D pointed out, pwd has the -P option to ignore $PWD.
[131069190060] |It also has the -L option to return the contents of $PWD.
[131069190070] |The man page for pwd does not say which option is used by default but experience tells me the above description is correct (shell pwd vs. /bin/pwd).
[131069190080] |However you should probably not rely on that and just use pwd -P.
[131069200010] |What does dd stand for?
[131069200020] |I know very well what the command does, but man dd, info dd tell me: 'Convert and copy a file', as does GNU Coreutils.
[131069200030] |Google says its an abbreviation of everything between medicine and bad webchat slang; except someone saying it means 'data destroyer', something used in PC forensics - I'd be horrified if my dd destroyed my data!
[131069200040] |Any insight? :-)
[131069200050] |Update: Of course I had to check the jargon file:
[131069200060] |The Unix dd(1) was designed with a weird, distinctly non-Unixy keyword option syntax reminiscent of IBM System/360 JCL (which had an elaborate DD ‘Dataset Definition’ specification for I/O devices)
[131069200070] |Still sounds pretty ambiguous, but then it says:
[131069200080] |though the command filled a need, the interface design was clearly a prank.
[131069200090] |Heh :-)
[131069210010] |Wikipedia (dd) asserts it was named after IBM JCL command DD which stands for Data Description.
[131069210020] |I always thought it would mean data duplicate, though.
[131069220010] |My guess: it has something to do with "Direct Disk access". :)
[131069230010] |"dump data".
[131069230020] |JCL is irrelevant.
[131069240010] |Though the best answer was given, this site states otherwise:
[131069240020] |Actually, it stands for Copy and Convert' and was renamed toddonly becausecc' was reserved for the C compiler!
[131069240030] |This is the authentic information I got from the man pages of our Unix-V7 on our university PDP 11.
[131069250010] |How can i check that a remote computer is online for ssh / script acess ?
[131069250020] |i'm writting a script that will backup data from my laptop to an ubuntu server. to do so, i'm looking for a (ba)sh command to test if the server is available before starting the backup. something like "ping" on port 22 that returns a boolean
[131069250030] |how can i do this ? thanks for helping
[131069260010] |Use netcat:
[131069260020] |From the manpage
[131069270010] |Like this:
[131069270020] |If it's 0 then it's available.
[131069270030] |If it's 1 then it's not.
[131069280010] |What about
[131069290010] |How do I turn on http compression?
[131069290020] |I'm running a CentOS server and I want to save bandwidth, make pages faster, all that.
[131069290030] |How can I turn on http compression?
[131069290040] |Is it a module I enable for all sites?
[131069290050] |Can it be that simple?
[131069300010] |Use mod_deflate.
[131069300020] |Add this to your apache config:
[131069300030] |Obviously if the path your system uses for apache modules differs then you'll need to use the correct path.
[131069310010] |Find incoming symlinks
[131069310020] |In Linux, what is the best way to find all symbolic links that point to a given file (regardless of whether the symlink is relative or absolute)?
[131069310030] |I realize this will require scanning the whole filesystem.
[131069320010] |Perhaps the shortest way would be:
[131069320020] |But it's not very efficient.
[131069320030] |If you don't have realpath, install it, e.g. apt-get install realpath.
[131069320040] |You could also use stat -N or ls -l or pwd -P to emulate realpath, but those ways are harder.
[131069320050] |Also, the above example won't handle file names with spaces in it properly.
[131069320060] |Here's a better way to do it.
[131069320070] |Note that IFS=$'\n' requires bash or zsh.
[131069330010] |GNU find has the -samefile test.
[131069330020] |According to the man page:
[131069330030] |This will find all links to /path/to/file, which includes hard links and the file itself.
[131069330040] |If you only want symlinks, you can individually test the results of find (test -L).
[131069330050] |You should read up on what the effects of -L are and ensure that it won't cause you any problems with your search.
[131069330060] |Note: While the documentation says it looks for files with the same inode number, it does appear to work across filesystems.
[131069330070] |e.g. /home and /tmp are separate filesystems
[131069330080] |Note how this is returning /tmp/bar, which is a symlink to /tmp/foo, which is a symlink to ~/testfile.
[131069330090] |If you only wanted to find direct symlinks to your target file, this wont work.
[131069340010] |How does a unix or linux system work?
[131069340020] |I would like to know how the OS works in a nutshell:
[131069340030] |The basic components it's built upon
[131069340040] |How those components work together
[131069340050] |What makes unix UNIX
[131069340060] |What makes it so different from other OSs like Windows
[131069350010] |"Design of the UNIX Operating System" would be good book to read.
[131069360010] |UNIX is a strong OS, build on a sound design that has proven successful for more than 40 years (that's almost eternity in computer science).
[131069360020] |The central technology is based on the C language and a myriad of small programs: the UNIX commands.
[131069360030] |The basic philosophy has been summarized by McIlroy:
[131069360040] |Write programs that do one thing and do it well.
[131069360050] |Write programs to work together.
[131069360060] |Write programs to handle text streams, because that is a universal interface.
[131069360070] |More on the UNIX philosophy can be found in E.S.Raymond "The Art of UNIX Programming".
[131069370010] |I would recommend reading Advanced Programming in a Unix Environment 2e to learn a lot about the Single Unix Standard (SUS) API and POSIX, which will give you an idea about what makes Unix Unix and how the components work, and work together.
[131069370020] |However, it's a very C heavy book and more of a reference manual.
[131069370030] |If you have a problem with insomnia just take it to bed with you.
[131069370040] |That aside if you are a Unix C programmer it's a must have.
[131069380010] |A UNIX system consists of several parts, or layers as I'd like to call them.
[131069380020] |To start a system, a program called the boot loader lives at the first sector of a hard disk partition.
[131069380030] |It is started by the system, and in turn it locates the Operating System kernel, and load it.
[131069380040] |Layering
[131069380050] |The Kernel.
[131069380060] |This is the central program which is started by the boot loader.
[131069380070] |It does the basic hardware interaction for the system (disk, memory, video, sound) and offers a virtual environment in which it can start programs.
[131069380080] |The kernel also ships all drivers which deal with all the little differences between hardware devices.
[131069380090] |To the outside world (the higher layers), each class of devices appear to behave exactly in the same consistent way - which in turn, the programs can build upon.
[131069380100] |Background subsystems.
[131069380110] |There are just regular programs, which just stay out of your way.
[131069380120] |They handle things like remote login, provide a cental message bus, and do actions based on hardware/network events.
[131069380130] |For example, bluetooth discovery, wifi management, etc..
[131069380140] |Any network services (file server, print server, web server) also live at this level.
[131069380150] |In UNIX systems, these are all just normal programs.
[131069380160] |The command line tools.
[131069380170] |These are all little programs which can be started to do things like text editing, downloading files, or administrating the system.
[131069380180] |At this point, a UNIX system is fully usable for system adminstrators.
[131069380190] |In Windows, this layer didn't not really exist anymore.
[131069380200] |The graphical user interface.
[131069380210] |These are also just programs, the only difference is they draw windows at the screen instead of writing text.
[131069380220] |This makes the system easier to use for regular users.
[131069380230] |Any service or event will go from the bottom all up to the top.
[131069380240] |Libraries - the common platform
[131069380250] |Programs do a lot of common things like displaying a window, drawing stuff at the screen or downloading a file.
[131069380260] |These things are the same for multiple programs, hence that code are put in separate "library" files (.so files - meaning shared object).
[131069380270] |The library can be shared across all programs.
[131069380280] |For every imaginable thing, there is a library.
[131069380290] |There is one for reading/writing PNG files.
[131069380300] |There is one for JPEG files, for reading XML, for encryption, for video playback, and so on.
[131069380310] |On Linux, the common libraries for application developers are Qt and Gtk.
[131069380320] |These libraries use lower-level libraries internally for their specific needs, while exposing their functionality in a nice consistent and concise way for application developers to create applications even faster.
[131069380330] |Libraries provide the application platform, on which programmers can build end user applications for an Operating System.
[131069380340] |The more high quality libraries a system provides, the fewer code a programmer has to write to make a beautiful program.
[131069380350] |Some libraries can be used across different operating systems (for instance, Qt is), some are really specifically tied into one operating system.
[131069380360] |This will restrict your program to be able to run at that platform only.
[131069380370] |Inter process communication
[131069380380] |A third corner piece of an operating system, is the way programs can communicate with each other.
[131069380390] |These are Inter Process Communication (IPC) machanisms.
[131069380400] |These exist in several flavors, e.g. a piece of shared memory, or a small channel is set up between two programs to exchange data.
[131069380410] |There is also a central message bus on which each program can post a message, and receive a response.
[131069380420] |This is used for global communication, where it's unknown which program can respond.
[131069380430] |From libraries to Operating Systems
[131069380440] |With libraries, IPC and the kernel in place, programmers can build all kinds of applications for system services, user administration, configuration, administration, office work, entertainment, etc..
[131069380450] |This forms the complete suite which novice users recognize as the "operating system".
[131069380460] |In UNIX/Linux systems, all services are just programs.
[131069380470] |All system admin tools are just programs.
[131069380480] |They all do their job, and they can be chained together.
[131069380490] |I've summarized a lot of major programs at http://codingdomain.com/linux/sysadmin/
[131069380500] |Distinguishable parts with Windows
[131069380510] |UNIX is mainly a system of programs, files and restricted permissions.
[131069380520] |A lot of complexities are avoided, making it a powerful system while it looks like it has an easy job doing it.
[131069380530] |In detail, these are principles which can be found across UNIX/Linux systems:
[131069380540] |There are uniform ways to access information.
[131069380550] |("Everything is just a file").
[131069380560] |You can open a file, network socket, IPC channel, kernel parameters and block device as a file.
[131069380570] |Hence the appearance of the virtual filesystems in /dev, /sys and /proc.
[131069380580] |The only API you ever need is open, read and close.
[131069380590] |The underlying system is transparent.
[131069380600] |Every program operates under the same rules.
[131069380610] |Unlike Windows, there is no artificial difference between a "console program", "gui program" or "background service".
[131069380620] |They are all just programs, that happen to do different things.
[131069380630] |They can also all be observed, analyzed and debugged in the same way.
[131069380640] |Settings are readable, editable, and can be annotated with comments.
[131069380650] |They typically have an INI-style format, but may use a custom format for the needs of that application.
[131069380660] |Because they are just files, they can be copied to other systems, archived or being backuped with standard tools.
[131069380670] |No large "do it all in once" applications.
[131069380680] |The mantra is "do one thing, do it well".
[131069380690] |Command line tools can be chained and together be powerful.
[131069380700] |Separate services (e.g. SMTP, IMAP and POP, and login) are separate subprograms, avoiding complex intertwined code and security issues.
[131069380710] |Complex desktop environments delegate hard work to individual programs.
[131069380720] |fork().
[131069380730] |New programs are started by an existing program cloning itself.
[131069380740] |The clone sets up everything (e.g. file handles), and optionally replaces itself with the new program code.
[131069380750] |This makes it really easy to apply the same security settings and restrictions to new programs, share memory or setup an IPC mechanism.
[131069380760] |The cost of starting a process is also very low.
[131069380770] |The file system is one tree, in which other disk partitions and network shares can be mounted.
[131069380780] |There is again, an universal way of accessing data.
[131069380790] |Common system locations (e.g. /usr can easily be mounted as network share.
[131069380800] |The system is built for low user privileges.
[131069380810] |After login, every user (except root) is confined their own resources, running applications and files only.
[131069380820] |Network services reduce their privileges as soon as possible.
[131069380830] |There is a single clear way to get more privileges, or ask someone to execute a privileged job on their behalf.
[131069380840] |Every other call is limited by the restrictions and limitations of the program.
[131069380850] |Every program stores settings in a hidden file/folder of the user home directory.
[131069380860] |No program ever attempts to write a global setting file.
[131069380870] |A favor towards openly described communication mechanisms over secret mechanisms or specific 1-to-1 mechanisms.
[131069380880] |Other vendors and software developers are encouraged to follow the same specification, so things can easily be connected, swapped out and yet stay loosely coupled.