Details on the memory extension (aufs)

Hi!

I'd like to know some details about aufs. If I understand correctly, it creates a union of two different locations (/media/disk and /home/user) into /mnt/home. Both of the source locations contain file ".wh..wh.aufs" and directory ".wh..wh.plink/", while the union does not.

I'd be interested in some details on how the union works, so if anyone here knows more about this, I'd really appreciate the information:

- How does aufs determine which source location to write the file on?
- Is there a way to force a file to be stored on the extension disk? (This is relevant, since the integrated 8GB disk is used for system files as well, and I'd like to keep large files away from it.)
- Will aufs ever automatically move files from a source location to another, for example if other source disk runs low on disk space?
- What happens if I accidentally detach the disk?
- Can the extension disk be used to transfer files to (and from) another computer?
- Are there any pitfalls I should be aware of, when using aufs?
- What happens if there's a file with the same name in both source directories?
- Can the source directories be used safely - for example is it ok to copy a file into a /media/disk to force it to be stored on the extension card? (This is an especially relevant question because the shell home directory is apparently one of the source paths, not the union, and not being aware of this I've already written files there directly. Also see my other post, "Changing the shell home directory to use memory extension".)

 

http://macles.blogspot.com/2008/09/abou ... spire.html should answer some of your questions.

Each drive/card has it's own location on the file system, as well as the "combined" location, so you can save specifically to each. I literally just got a 16GB card to use yesterday, so I might see if I can answer some of your other questions once I've experimented myself!

 

Thanks! That's was a good read.

So, to answer some of my own questions based on the document:

First of all, aufs only determines where to write files when you write them to the union directory (/mnt/home).

On Aspire One, aufs uses a policy called "mfs", to determine where to write. It always writes file to drive with Most Free Space, so usually only memory extension card gets written to. There are also other policies available in aufs, of which "tdp" may be interesting for Aspire One users. Using "tdp", files always get written to first branch, and to the second only if the first becomes full.

Yes, simply by saving the file into the chosen source path (/media/disk for extension card).

The files on the disk become unavailable, but appear again when the disk is re-inserted. I'd imagine the effect would be more fatal if the drive was being written to. So I'll add a question:
- How to tell Aspire One to make the extension card safe to be detached?

Yes.

 

Probably umount it, just like any other storage device.

Cheers.

 

The thunar file manager has two modes, standard and advanced, and in advanced the SD card is split back up and displayed as a removable drive, with an unmount button.

It seems absolutely fine to use the source directories. For example, I've set up transmission to go to downloads via media/disk, and openoffice to go to documents via that path. My plan is to try to keep docs, videos, music etc on the card, and let the system go mad with the SSD, so I can do a system backup of the SSD every now and again, without copying 4GB worth of mp3s. I'm not sure how achievable this will be. However, I've successfully copied my docs etc from /home/user to media/disk, and they're happily turning up in mnt/home as if nothing had changed.

There's another comment by macles: "It will only distribute files among both branches if the parent directory exists on both, which is the case for the five default directories in the home directory. If you manually create a directory on the SD then all files will only be copied into that directory and not distributed among both branches, even though it's also visible in the home directory of the file browser." So this rather implies that if I delete Downloads from home/user, but leave it in media/disk, then /mnt/home/Downloads will always go to the SD card, or if I move .thunderbird to media/disk, it will keep my emails on the SD card. Alternatively, it will break something horribly. It also implies that a lot of system stuff will simply stay on the SSD forever, and the overall free space will be an illusion in some senses - which is another reason for me to shift docs onto the card.

To answer another of your questions, I copied a file into both real drives, and it only showed up once in the virtual. I then deleted it from one of them, and the other one replaced it! Very interesting behaviour.
I don't know the answer to moving files automatically, but I suspect not, based on what I've read so far.

Anyhow, I hope that's of help - please post if you find anything else out.

 

Yes, I've also now done some experimenting, and haven't encountered any problems with writing (different files) to direct paths. Wothy of noting is that that there sometimes appears to be a short delay (around 5 seconds) before the changes appear in the union path.

However, problems appear when you create a file with the same name to both sources. Apparently, when you first access a file thourgh the union, the union locks itself to that particular file, and ignores the other source after that. If, however, the file is deleted, then the union checks whether the file exists in the other source, and locks on to that. This works even when done via the union, so "rm unionpath/duplicate.file" removes only the one it has locked onto and thus makes the other one appear instead of it, seeming as if the file wasn't deleted.

The bad part is, this behaviour seems to extend to directories as well. When you create a directory to source A and access it through the union, it gets locked to that source. If you then create the same directory to source B, the union does not display contents of both directories as you might have expected - it continues to display only contents of source A, which it has locked onto.

However, if you create the directory on both sources and access the directory through union only after both directories exist, then the union does correctly display the contents of both directories.

Writing to the directory though the union seems even more trickier. If the directory is only in source A, but source B is the one the union want to write to (because it has more space), when you write to the union directory, it creates the same directory in B, and writes there (afterwards correctly displaying contents of both directories, of course). However, if you create the directory manually to source B, this creates a similar situation as I described above - only source A is displayed. Now when you try to write to the union directory, you actually get an Input/Output error, as the aufs attempts to create the directory, but fails as the directory already exists. Note that this only happens if the union wants to write to the source it's not locked on to - if the union has locked on to the directory on the preferred source (source that has more space), the union writes correctly.

I think this is actually very bad behaviour from the aufs! What I'd expect it to do, is to always display the union of contents in both source directories, regardless of how I've created them. As for duplicate files, that would be a trickier problem to handle.

Anyways, I'm planning on writing a checking script to at least detect any duplicate files (that is, compare the directory srtucture of /home/user and /media/disk, and output any duplicate entries). Since aspire one shell defaults to /home/user, which is a source, not the union, and file manager uses the union, that uses /media/disk as the preferred source, it might actually be quite easy to create accidental duplicates! (Any help on writing the checkup script is appreciated.)

According to my experiments, this would not be true. If you do manually create a directory on source A, and the union prefers source B, the union will create the same directory on source B and place file there, when you write to the union.

Well, this would be correct as long as /media/disk remains the preferred source of aufs. If it is not, aufs will apparently then create the required directory structure on /home/user, and write the files there.

 

Hi all

This thread gave me some inspiration but still left me confused about SD card in the left slot.

I am new to Linux/Linpus and have grown accustomed to the good old windows directories, A, C, D etc. therefore I certainly find the Linpus ones quite confusing and particularly when an SD card is added to the left slot.
It appeared to me that everything I saved was going to the SSD and nothing to the SD. I began to wonder what was the point of the left hand SD other than to show that I now had more space available.
Then I found reference to Auf's and that confused me even more. I understand the Aspire default is “mfs” and that all data should be written to the physical directory with the greatest free space. As you will see from my signature below this should be the SD card, but that was not happening for me.
Initially I could not find the SD card on File Manager, then I discovered Tree view on the Side Pane of FM. This produced a list of what looked like the Windows directory structure. The following is that structure after I had carried out the steps below and in my case included;

home is know as My Disk:/// and is directory /mnt/home a Virtual directory
Trash
SD/MMC Drive is know as Removable Disk :/// and is directory /media/disk an SD card and physical directory
G Removable Volume a USB Pen if mounted
File System More of this later

Under both home and SD/MMC the folder structure automatically created was the same 7 folders and I assumed that home was /home/user as it was also called My Disk /// (SSD?).
I then decided to follow procedure in macles blog to set up the auf's as “tdp” but that did not appear to help me as I still had the same file structure containing identical folder names with the same data other than that I had pasted into SD.
I found this rather confusing so I renamed the Documents folder under SD/MMC to Documents SD. I was rather surprised when I again looked at the 2 directories and found they both now contained a Documents and a Documents SD folder.

This led me to explore the File System and in this structure you can find the contents of the directories /mnt/home, /media/disk and /home/user. This showed that home above was the virtual directory made up of both physical directories and not as I had first assumed. The /home/user did not contain the Documents SD folder I had created by renaming. The /home/user is the other physical directory.

When I unmount the SD card any reference to it and its contents are removed from home and if I then mount it they are all returned.

Slightly off subject but related to where you save data. When using Save As in Open Office the default option is my My Disk (SSD) and its associated Folders. No problem if you do not have an SD card installed. If you do and you want to save to it you can find it by browsing for other folders and selecting the Media folder (i.e. SD card). You can then ADD it to the default list.

The above is my trial and error approach to establish a means of ensuring that I write data to the SD card rather than SSD and to better understand the folder/file structure of the Aspire One. I apologise if it sounds silly to any Linux/Linpus/Aspire One expert.

I hope it is of some use to beginners like me.

 

First of all, the left side slot is meant to be used as a permanent memory extension, that's why it gets automatically hidden and merged with the standard system. The idea is that when you install a memory extension card, you simply automatically get more space, and don't have to care about where you are writing data. If you don't ever go poking around in the file system or shell, and only use your home directory, you don't have to care about /media/disk, /mnt/home and such. That said, if you do go exploring, it gets quite confusing as the system seems to be implemented rather poorly. The paths don't really make much sense, behaviour on manipulating the sources can be quite erratic (as explained above), and so on. The correct place to write files should be /home/user as it is in other linux systems, but in AA1 it's actually the rather nonsensical /mnt/home.

The idea behind the thing is awesome, but the implementation is poor. I'm not sure this it's a good idea from acer to assume that users should just stick to the home directory. The source paths should have a more logical structure (something like /mnt/internal, /mnt/external, and /home/user as the union) - that way it would be easy to always just ignore the underlying aufs mechanics, and keep on using the correct path. However, with this implementation everything looks really weird once you look even a bit under the hood of the system.

Anyways, the idea is that use the left side slot only for permanent memory extension, and always write to /mnt/home (which is file manager's home directory). If you want to tranfer files to an external disk, the right side slot works as it should - it appears at /media/something (and also as a direct path in file manager), and there's no weird merging of any locations to worry about.

 

Hi ikajaste

You are right when you say “if you do go exploring, it gets quite confusing”

When an SD card is inserted in the left hand slot I do not consider it is hidden as it depends which view you use in File Manager. If you use “View” then “Side Pane” then select “Tree” most installed disks , external and internal, are shown. The one that is not immediately shown is /home/user but it can be found under “File System”. Now I assume ACER defaulted the top of the tree to the Virtual Disk /mnt/home as it will contain anything that is input to the other 2 physical disks there by allowing for users who insert a card in the left slot. There may be other technical reasons Acer have done this and macles blog on “auf's” suggest s there are advantages in storing data on the SD to let the SSD get on with processing work. My technical knowledge does not allow me to say if this is real or imaginary.

As you say users may not care, particularly ones without a left side SD card, as long as they can use all the physical disc space they have installed.

Interestingly if you mount another SD card in the right hand slot it is also displayed in the tree as SD/MMC Drive and is also contained in the /media/disk folder within File System but is called Disk-1. This is true if the left hand SD is there first otherwise the Disk and Disk-1 are reversed.

Best wishes

 

Hi,

I'd like to mimic the memory expansion on a Hardy netbook remix install. Any ideas on how to go about this? Can we trace what is happening in sequence when a sdhc card is inserted in the left slot under Linpus? Are there any usable scripts?

Thanks,

-stu

 

The program aufs is simply a separately developed service that Acer plugged into their version of linpus, so in theory there should be nothing to stop you installing it on a different distro. I'd imagine the devil will be in the details, though

 

As markh pointed, aufs is a generic linux program/service/filesystem/something, and can be installed on Ubuntu. I think a good way to start is to read up on aufs documentation, and first try to get aufs working on two test folders.

While poking around my AA1, I did come across some scripts that seem to be used for the automatic aufs mounting and file system creation, and they even had some comment lines as documentation - unfortunately I don't right now remeber where they were on the system. I think it could be possible to mimic the activation process by looking at those scripts, armed with some expert knowledge or a good relationship with documentation texts (neither of which I happen to possess )

 

Ah, true. They've indeed done a really incomplete job of hiding the underlying mechanic. The fact that the disk disappears from File Manager in default mode suggest that that was their intention. But I agree in that I wouldn't count the Tree mode as advanced, so they've really screwed it up.

This indeed goes to prove that /media/disk is a really illogical location, if the intention is that users do not use it as a removable media. It sounds like the haven't gone through the trouble to make the Storage Extension slot mount into a different location, and have just used the default because it's easier.

 

How can I disable aufs entirely (but still be able to use the slot to mount a card)?