| View previous topic :: View next topic |
| Author |
Message |
The Natural Philosopher
Guest
|
 Posted: Wed Nov 19, 2008 6:06 pm Post subject: Re: disk spin down with lots of log files |
 |
|
Hal Murray wrote:
| Quote: | By Murphey's law, the system would crash just before your hours of log data
was written to disk, so you would have nothing to debug with.
I'm prepared to take that risk. The main reason I miss data currently
is because I'm hacking with the software.
If a system is only used for something simple, it's easy to have
uptimes of many months. That box hasn't had any problems since
a month ago when I updated the kernel. Another box that's not
doing much has been up for 3 months.
|
well, hack syslogd then, and make a custom one: shouldn't ne ard to
store a bunch of stuff in its own internal data structures and spit it
out and synch() it every couple of hours..
| Quote: |
If you have two or more computers on a LAN, why not have each machine keep
its log files on the other?
I'm trying to reduce power consumption, not double it.
|
|
|
| |
|
 Back to top |
Cork Soaker
Guest
|
| Posted: Wed Nov 19, 2008 8:21 pm Post subject: Re: disk spin down with lots of log files |
|
|
Hal Murray wrote:
| Quote: | If you have two or more computers on a LAN, why not have each machine keep
its log files on the other?
I'm trying to reduce power consumption, not double it.
|
You know this will reduce the life of the hard drives significantly
don't you?
I'm all for power management though...
Make it sync once a week.  |
|
| |
|
| Back to top |
Cork Soaker
Guest
|
| Posted: Wed Nov 19, 2008 8:27 pm Post subject: Re: disk spin down with lots of log files |
|
|
Hal Murray wrote:
| Quote: | Could mount the flash card as /var/log/
Generally, flash cards don't like lots of small writes.
|
You could use JFFS, a ram disk and then set the ram disk to copy the
files to the flash disk when it reaches a certain size. This would
leave the hard drive alone altogether, and shouldn't wear out the flash
disk too much.
The flash disks wear out after multiple writes to the same sector, not
to the disk generally. I *think* JFFS uses random block selection to
reduce this. One of the file systems do anyway, can't remember which one. |
|
| |
|
| Back to top |
Jean-David Beyer
Guest
|
| Posted: Wed Nov 19, 2008 9:29 pm Post subject: Re: disk spin down with lots of log files |
|
|
Hal Murray wrote:
| Quote: | By Murphey's law, the system would crash just before your hours of log data
was written to disk, so you would have nothing to debug with.
I'm prepared to take that risk. The main reason I miss data currently
is because I'm hacking with the software.
If a system is only used for something simple, it's easy to have
uptimes of many months.
|
Absolutely. The record for some Linux systems exceeds a year. My record was
over 6 months running Red Hat Linux 7.3. (My present machine is not up much
lately because I have 8 memory modules and (at least) one seems bad. It gets
an ECC error every few days. I have removed all but two of the modules now,
and it has been up almost a week, so I guess those are good. Memtest-86
reports them all as good, but I cannot run it for a week.
| Quote: | That box hasn't had any problems since
a month ago when I updated the kernel. Another box that's not
doing much has been up for 3 months.
If you have two or more computers on a LAN, why not have each machine keep
its log files on the other?
I'm trying to reduce power consumption, not double it.
I did not say to buy another computer. I suggested using one you already |
have (if you have it).
--
.~. Jean-David Beyer Registered Linux User 85642.
/V\ PGP-Key: 9A2FC99A Registered Machine 241939.
/( )\ Shrewsbury, New Jersey http://counter.li.org
^^-^^ 10:25:01 up 6 days, 19:33, 4 users, load average: 4.18, 4.30, 4.42 |
|
| |
|
| Back to top |
Cork Soaker
Guest
|
| Posted: Wed Nov 19, 2008 9:49 pm Post subject: Re: disk spin down with lots of log files |
|
|
Jean-David Beyer wrote:
| Quote: | If you have two or more computers on a LAN, why not have each machine
keep its log files on the other?
I'm trying to reduce power consumption, not double it.
I did not say to buy another computer. I suggested using one you already
have (if you have it).
|
You've missed the point completely. |
|
| |
|
| Back to top |
Jean-David Beyer
Guest
|
| Posted: Wed Nov 19, 2008 10:48 pm Post subject: Re: disk spin down with lots of log files |
|
|
Cork Soaker wrote:
| Quote: | Jean-David Beyer wrote:
If you have two or more computers on a LAN, why not have each machine
keep its log files on the other?
I'm trying to reduce power consumption, not double it.
I did not say to buy another computer. I suggested using one you already
have (if you have it).
You've missed the point completely.
|
I guess I did. And I still do. How much power do you suppose it takes to
send your log file appends over a LAN? Surely not double the power a
computer uses.
--
.~. Jean-David Beyer Registered Linux User 85642.
/V\ PGP-Key: 9A2FC99A Registered Machine 241939.
/( )\ Shrewsbury, New Jersey http://counter.li.org
^^-^^ 11:45:01 up 6 days, 20:53, 4 users, load average: 4.42, 4.33, 4.27 |
|
| |
|
| Back to top |
Cork Soaker
Guest
|
| Posted: Wed Nov 19, 2008 11:05 pm Post subject: Re: disk spin down with lots of log files |
|
|
Jean-David Beyer wrote:
| Quote: | Cork Soaker wrote:
Jean-David Beyer wrote:
If you have two or more computers on a LAN, why not have each
machine keep its log files on the other?
I'm trying to reduce power consumption, not double it.
I did not say to buy another computer. I suggested using one you
already have (if you have it).
You've missed the point completely.
I guess I did. And I still do. How much power do you suppose it takes to
send your log file appends over a LAN? Surely not double the power a
computer uses.
|
You're still missing the point.
Try to re-read the original post carefully. |
|
| |
|
| Back to top |
Jean-David Beyer
Guest
|
| Posted: Thu Nov 20, 2008 12:37 am Post subject: Re: disk spin down with lots of log files |
|
|
Cork Soaker wrote:
| Quote: | Jean-David Beyer wrote:
Cork Soaker wrote:
Jean-David Beyer wrote:
If you have two or more computers on a LAN, why not have each
machine keep its log files on the other?
I'm trying to reduce power consumption, not double it.
I did not say to buy another computer. I suggested using one you
already have (if you have it).
You've missed the point completely.
I guess I did. And I still do. How much power do you suppose it takes to
send your log file appends over a LAN? Surely not double the power a
computer uses.
You're still missing the point.
Try to re-read the original post carefully.
|
I tried and failed.
--
.~. Jean-David Beyer Registered Linux User 85642.
/V\ PGP-Key: 9A2FC99A Registered Machine 241939.
/( )\ Shrewsbury, New Jersey http://counter.li.org
^^-^^ 13:35:01 up 6 days, 22:43, 4 users, load average: 4.35, 4.33, 4.27 |
|
| |
|
| Back to top |
Hal Murray
Guest
|
| Posted: Thu Nov 20, 2008 2:42 am Post subject: Re: disk spin down with lots of log files |
|
|
| Quote: | I guess I did. And I still do. How much power do you suppose it takes to
send your log file appends over a LAN? Surely not double the power a
computer uses.
|
The case I'm interested in is running off a UPS.
In general, home size UPS boxes don't last very long. It's plenty
of time to ride over short glitches and long enough to shut down
cleanly if the power is seriously out, but that's minutes rather than hours.
One of the things I want to record is how long the power is out, so
I'd like my setup to last as long as it can.
(I'm also playing the game of make it low power, or trying to, I
don't seem to be getting very far.)
--
These are my opinions, not necessarily my employer's. I hate spam. |
|
| |
|
| Back to top |
Jean-David Beyer
Guest
|
| Posted: Thu Nov 20, 2008 3:52 am Post subject: Re: disk spin down with lots of log files |
|
|
Hal Murray wrote:
| Quote: | I guess I did. And I still do. How much power do you suppose it takes to
send your log file appends over a LAN? Surely not double the power a
computer uses.
The case I'm interested in is running off a UPS.
In general, home size UPS boxes don't last very long. It's plenty
of time to ride over short glitches and long enough to shut down
cleanly if the power is seriously out, but that's minutes rather than hours.
One of the things I want to record is how long the power is out, so
I'd like my setup to last as long as it can.
|
All my UPSs are APC Smart-UPS models. I run them with the apcupsd package.
When the power goes down, it logs that. Then it runs until the battery is
almost exhausted. It logs that and shuts down the computer. So the machine
does not crash due to power failure. Then when the power returns, it logs
when it reboots the machine. Depending on the configuration, it may wait a
while or two before rebooting. There are two delays, both user configurable.
One is a minimum time after power is restored before it tries to reboot.
This is so that it does not try to reboot when the power is flicking on and
off. The other is the minimum battery charge level you specify before it
reboots.
Of course, if logging is turned off, you may not see this. However, it
records this in file /var/log/apcupsd.events, so you could turn off all the
other logging if you like and your hard drives would be more likely to go
into power saving mode. The instant of power failure would cause a logging
event to be posted into the apcupsd.events file and would restore your hard
drive to normal operation until the power actually was cut off by the UPS.
| Quote: |
(I'm also playing the game of make it low power, or trying to, I
don't seem to be getting very far.)
How about getting a stopwatch. Turn off the circuit breaker and start the |
stopwatch. When the machine shuts down, stop the stopwatch. That way you
could time it at your convenience and would not have to change anything.
--
.~. Jean-David Beyer Registered Linux User 85642.
/V\ PGP-Key: 9A2FC99A Registered Machine 241939.
/( )\ Shrewsbury, New Jersey http://counter.li.org
^^-^^ 16:30:01 up 1:58, 4 users, load average: 4.22, 4.23, 4.24 |
|
| |
|
| Back to top |
Darren Salt
Guest
|
| Posted: Fri Nov 21, 2008 6:22 am Post subject: Re: disk spin down with lots of log files |
|
|
I demand that Jean-David Beyer may or may not have written...
[snip]
| Quote: | The record [uptime] for some Linux systems exceeds a year. My record was
over 6 months running Red Hat Linux 7.3.
|
One of mine has been up for over 506 days. I expect to reboot it for a kernel
upgrade or to shut it down due to some hardware failure; however, a
sufficiently long power failure will also cause a shutdown (yes, UPS).
| Quote: | (My present machine is not up much lately because I have 8 memory modules
and (at least) one seems bad. It gets an ECC error every few days. I have
removed all but two of the modules now, and it has been up almost a week,
so I guess those are good. Memtest-86 reports them all as good, but I
cannot run it for a week.
|
Another of mine was last rebooted due to a kernel panic somewhere in the
networking code (2.6.25.* kernel; now running 2.6.27.*). Its uptime is
currently 2½ days.
[snip]
--
| Darren Salt | linux or ds at | nr. Ashington, | Toon
| RISC OS, Linux | youmustbejoking,demon,co,uk | Northumberland | Army
| + Buy local produce. Try to walk or cycle. TRANSPORT CAUSES GLOBAL WARMING.
Smash forehead against keyboard to continue... |
|
| |
|
| Back to top |
Hal Murray
Guest
|
| Posted: Fri Dec 05, 2008 5:07 pm Post subject: Re: disk spin down with lots of log files |
|
|
Thanks for all the suggestions.
Now that I've got it working, it was probably a wild goose chase. It only
saves a watt or so out of 18 or 19.
Here are my notes in case anybody else wants to do the same thing:
I added /etc/rc.d/init.d/no-spin
to start and stop things.
I added a cron job to backup the data.
I put the scripts that actually do the work in /etc/no-spin/
start does:
mount -t tmpfs -o size=100m tmpfs /var/log2
then makes several directories, chmod/chown as needed
and copies in a few recent versions of the old log files from /var/log/
backup does an rsync from /var/log2/ to /var/log/
since the output gets mailed to me, it does a df and du too.
stop just calls backup
I've got backup running every 4 hours.
I haven't done anything about deleting old log file. I'll probably
do that by hand.
Then I had to fixup several programs to put their stuff into /var/log2/
instead of /var/log/ or /home/~/xxx
There were a couple of quirks I hadn't anticipated.
I had a sudo step that went off every 5 minutes (to read the disk temp via
hdparm). That wrote to /var/log/secure I "fixed" that sudo entry to not
log anything. (I could have run that job as root.)
cron was adding hourly log entries to /var/log/cron Since that wasn't doing
anything, I commented out the hourly cron runs.
Now the disk is mostly off.
It still spins up when I ssh in or use scp to grab the log files.
/var/log/{wtmp,secure,lastlog} get updated.
I had to fix the script that grabs some log files too. It lives on another
machine.
It's not great, but it's good enough for now.
What I'd really like is something like a mount option that says "Don't write
anything to disk, let it sit in the buffer cache". Then backup would be just
turn that bit off and do a sync.
Or a magic mount option that makes a shadow directory a copy-on-write copy of
the directory underneath it.
--
These are my opinions, not necessarily my employer's. I hate spam. |
|
| |
|
| Back to top |
|
