An open community 
of Macintosh users,
for Macintosh users.

FineTunedMac Dashboard widget now available! Download Here

Topic Options
#6255 - 12/02/09 06:11 AM shorter maintenance script
jchuzi Offline


Registered: 08/04/09
Loc: New York State
In OSs previous to 10.6, running the Weekly maintenance script took several minutes. In 10.6, by contrast, it only takes a few seconds. (The Daily and Monthly scripts still take just a few seconds.) Any ideas as to why this has changed?
_________________________
Jon

OS 10.14.2, iMac Retina 5K 27-inch, late 2014, 3.5 GHz Intel Core i5, 1 TB fusion drive, 16 GB RAM, Epson SureColor P600, Photoshop CC, Lightroom CC, MS Office 365

Top
#6267 - 12/02/09 11:12 AM Re: shorter maintenance script [Re: jchuzi]
Hal Itosis Offline


Registered: 09/03/09
Loc: 10.6.8 (build 10K549)
Originally Posted By: jchuzi
In OSs previous to 10.6, running the Weekly maintenance script took several minutes. In 10.6, by contrast, it only takes a few seconds. (The Daily and Monthly scripts still take just a few seconds.) Any ideas as to why this has changed?

No idea and no Snow.

But here are some commands we can use to compare (i have Leo Jr.):

This command lists the (weekly) scripts that get run:
ls -l /etc/periodic/weekly
total 24
-r-xr-xr-x 1 root wheel 635 Sep 28 2007 310.locate
-r-xr-xr-x 1 root wheel 1007 Sep 28 2007 320.whatis
-r-xr-xr-x 1 root wheel 620 Sep 28 2007 999.local



This command lists (all) the output files:
ls -lhrtT /var/{log/*.out,db/locate*}
-r--r--r-- 1 nobody wheel 8.0M Nov 28 03:24:52 2009 /var/db/locate.database
-rw-r--r-- 1 root wheel 1.6K Nov 28 03:25:11 2009 /var/log/weekly.out
-rw-r--r-- 1 root wheel 421B Dec 1 05:30:00 2009 /var/log/monthly.out
-rw-r--r-- 1 root wheel 35K Dec 2 03:15:03 2009 /var/log/daily.out



This (simplified) command looks at the last output (more-or-less):
tail /var/log/weekly.out
Sat Nov 28 03:15:00 EST 2009

Rebuilding locate database:

Rebuilding whatis database:
makewhatis: /usr/share/man//usr/share/man:/usr/X11/man: No such file or directory
makewhatis: /usr/X11/man//usr/share/man:/usr/X11/man: No such file or directory

-- End of weekly output --



E.g., does the 1st command produce 3 scripts of comparable sizes?
        does the 2nd command produce 4 files of comparable sizes?
        does the 3rd command produce anything?

Depending on the above results, more commands could explore the matter further.

Top
#6274 - 12/02/09 01:01 PM Re: shorter maintenance script [Re: Hal Itosis]
jchuzi Offline


Registered: 08/04/09
Loc: New York State
Thanks for responding, Hal. Here's what I get:

ls -l /etc/periodic/weekly
total 0
-r-xr-xr-x 1 root wheel 1008 May 18 2009 320.whatis
-r-xr-xr-x 1 root wheel 620 May 18 2009 999.local

ls -lhrtT /var/{log/*.out,db/locate*}
-rw-r--r-- 1 root wheel 3.1K Dec 1 20:27:35 2009 /var/log/daily.out
-rw-r--r-- 1 root wheel 174B Dec 1 20:27:43 2009 /var/log/weekly.out
-rw-r--r-- 1 root wheel 344B Dec 1 20:27:45 2009 /var/log/monthly.out

tail /var/log/weekly.out
Rebuilding whatis database:

-- End of weekly output --

Tue Dec 1 20:27:36 EST 2009

Rebuilding whatis database:

-- End of weekly output --
_________________________
Jon

OS 10.14.2, iMac Retina 5K 27-inch, late 2014, 3.5 GHz Intel Core i5, 1 TB fusion drive, 16 GB RAM, Epson SureColor P600, Photoshop CC, Lightroom CC, MS Office 365

Top
#6299 - 12/02/09 10:40 PM Re: shorter maintenance script [Re: jchuzi]
Hal Itosis Offline


Registered: 09/03/09
Loc: 10.6.8 (build 10K549)
Originally Posted By: jchuzi
Thanks for responding, Hal. Here's what I get:

ls -l /etc/periodic/weekly
total 0
-r-xr-xr-x 1 root wheel 1008 May 18 2009 320.whatis
-r-xr-xr-x 1 root wheel 620 May 18 2009 999.local

Hmm, well clearly that folder is missing the "310.locate" script. That's the time-consuming part of the weekly run (it builds a database of pathnames to found items... sorta like a Spotlight index, but with pathnames only).

So there's the answer to why yours runs so fast. (nothing [lengthy] to run)

Not having Snowy myself, i can't check to see if there should be a script there.

PAGING ALL 10.6 USERS READING THIS...

Please list this folder with Terminal (or take a peek in Finder via "Go To..." and tell us what's there): 
ls -l /etc/periodic/weekly

--

Some (possibly time-wasting) things to try until another 10.6 user joins us are:

type -a locate{,.updatedb}
locate is /usr/bin/locate
locate.updatedb is /usr/libexec/locate.updatedb


sed '/# Weekly /,/# Monthly /!d' /etc/defaults/periodic.conf
# Weekly options

# These options are used by periodic(8) itself to determine what to do
# with the output of the sub-programs that are run, and where to send
# that output. $weekly_output might be set to /var/log/weekly.log if you
# wish to log the weekly output and have the files rotated by newsyslog(8)
#
weekly_output="/var/log/weekly.out"
weekly_show_success="YES"
weekly_show_info="YES"
weekly_show_badconfig="NO"

# 310.locate
weekly_locate_enable="YES"

# 320.whatis
weekly_whatis_enable="YES"

# 999.local
weekly_local="/etc/weekly.local"

# Monthly options



I.e., does command #1 reveal any of Leopard's (10.5) "locate" executables?
       does command #2 reveal that periodic.conf has enabled the locate task?

But admittedly -- without that NNN.locate script existing somewhere -- there's nothing to call.

--

Ironically, if you had a locate database we could use locate to locate the locate script: wink
locate *[0-9][0-9][0-9].locate
/private/etc/periodic/weekly/310.locate

--

EDIT: but there's always regular Unix find...

find -x -f /private /usr -iname *.locate 2>/dev/null

[try that one jon]

EDIT: i tweaked ^it again (to check the /usr folder too).

I'm gonna step out on a limb here and bet that Apple probably removed that locate script in 10.6, figuring that anyone who *actually* needs it is probably "hip" enough to rig it for themselves. [maybe, idunno]


Edited by Hal Itosis (12/02/09 11:08 PM)
Edit Reason: i <3 editing

Top
#6307 - 12/03/09 03:16 AM Re: shorter maintenance script [Re: Hal Itosis]
jchuzi Offline


Registered: 08/04/09
Loc: New York State
Here we go:

type -a locate{,.updatedb}
locate is /usr/bin/locate
/usr/bin/type: line 4: type: locate.updatedb: not found

sed '/# Weekly /,/# Monthly /!d' /etc/defaults/periodic.conf
d: Event not found.

locate *[0-9][0-9][0-9].locate
locate: No match.
(Just as you said, Hal.)

find -x -f /private /usr -iname *.locate 2>/dev/null
find: No match.




_________________________
Jon

OS 10.14.2, iMac Retina 5K 27-inch, late 2014, 3.5 GHz Intel Core i5, 1 TB fusion drive, 16 GB RAM, Epson SureColor P600, Photoshop CC, Lightroom CC, MS Office 365

Top
#6321 - 12/03/09 09:51 AM Re: shorter maintenance script [Re: jchuzi]
Hal Itosis Offline


Registered: 09/03/09
Loc: 10.6.8 (build 10K549)
Okay, will all that evidence i think my "bet" (last sentence, previous post) is pretty safe.

Eventually some 10.6 user may confirm that. [or -- if you hang on until Xmas -- i'll probably pick up a new Mac and confirm it myself. wink ]

Top
#6323 - 12/03/09 10:02 AM Re: shorter maintenance script [Re: jchuzi]
Hal Itosis Offline


Registered: 09/03/09
Loc: 10.6.8 (build 10K549)
Originally Posted By: jchuzi
sed '/# Weekly /,/# Monthly /!d' /etc/defaults/periodic.conf
d: Event not found.

That error seems odd -- as if the hard (single) quotes didn't work.
So i'm wondering about your shell.  Can you show me this please:

echo "$SHELL $COMMAND_MODE"

thanx [i get: /bin/bash unix2003]

Top
#6324 - 12/03/09 10:09 AM Re: shorter maintenance script [Re: Hal Itosis]
dkmarsh Offline

Moderator

Registered: 08/04/09

I'm betting Jon is still running tcsh. shocked
_________________________

dkmarsh • member, FineTunedMac Co-op Board of Directors

Top
#6328 - 12/03/09 10:24 AM Re: shorter maintenance script [Re: dkmarsh]
Hal Itosis Offline


Registered: 09/03/09
Loc: 10.6.8 (build 10K549)
Originally Posted By: dkmarsh

I'm betting Jon is still running tcsh. shocked

No clean install since Jaggy? shocked shocked
laugh

Top
#6330 - 12/03/09 10:48 AM Re: shorter maintenance script [Re: Hal Itosis]
jchuzi Offline


Registered: 08/04/09
Loc: New York State
Both you and dk win.

echo "$SHELL $COMMAND_MODE"
/bin/tcsh unix2003

The only clean install that I did in OS X was Jag. All others have been via Upgrade.
_________________________
Jon

OS 10.14.2, iMac Retina 5K 27-inch, late 2014, 3.5 GHz Intel Core i5, 1 TB fusion drive, 16 GB RAM, Epson SureColor P600, Photoshop CC, Lightroom CC, MS Office 365

Top
#6334 - 12/03/09 12:56 PM Re: shorter maintenance script [Re: jchuzi]
Hal Itosis Offline


Registered: 09/03/09
Loc: 10.6.8 (build 10K549)
If you (or anyone else who needs to) would like to be Bourne Again:

sudo dscl . -create /Users/$USER UserShell /bin/bash

[that's a bit deeper than just tweaking Terminal's prefs.]

PS: by hanging on to tcsh (without a real good reason), you simply invite incompatibilities (like that silly quoting error we just saw two posts back) when interacting with the rest of the CLI community. All quickie commands posted these days use bash, and folks who post such one-liners do not bother to check if they would fail when run under a tcsh interpreter. [i know i don't]


Edited by Hal Itosis (12/03/09 01:50 PM)
Edit Reason: both parts of that (ancient) article are linked to

Top
#6336 - 12/03/09 02:49 PM Re: shorter maintenance script [Re: Hal Itosis]
jchuzi Offline


Registered: 08/04/09
Loc: New York State
Thanks, Hal. I did it and so far, at least, the computer hasn't blown up. grin

Now I get:

/bin/bash unix2003


Edited by jchuzi (12/03/09 02:51 PM)
_________________________
Jon

OS 10.14.2, iMac Retina 5K 27-inch, late 2014, 3.5 GHz Intel Core i5, 1 TB fusion drive, 16 GB RAM, Epson SureColor P600, Photoshop CC, Lightroom CC, MS Office 365

Top
#6341 - 12/03/09 05:24 PM Re: shorter maintenance script [Re: Hal Itosis]
roger Offline


Registered: 08/04/09
Loc: Vermont
here's what I got:
Code:
rogers-macbook:~ roger$ ls -l /etc/periodic/weekly
total 0
-r-xr-xr-x  1 root  wheel  1008 May 18  2009 320.whatis
-r-xr-xr-x  1 root  wheel   620 May 18  2009 999.local


I'm running 10.6.2
hope that helps!


Edited by roger (12/03/09 05:25 PM)
Edit Reason: added OSX version
_________________________
MacBook 2.4 Ghz · 4 Gb ram · 10.7.5
stuff I'm interested in
iPhone 4s 7.0.2

Top
#6355 - 12/04/09 09:48 AM Re: shorter maintenance script [Re: roger]
Hal Itosis Offline


Registered: 09/03/09
Loc: 10.6.8 (build 10K549)
Originally Posted By: jchuzi
Thanks, Hal. I did it <snip>
Now I get:
/bin/bash unix2003

Good man... welcome to the club. [Baptisms are performed the 2nd Tuesday of every month.]

Originally Posted By: jchuzi
and so far, at least, the computer hasn't blown up. grin

Perfect. (i configured it so no explosions occur until a few days have passed). wink



Originally Posted By: roger
here's what I got:
Code:
rogers-macbook:~ roger$ ls -l /etc/periodic/weekly
total 0
-r-xr-xr-x  1 root  wheel  1008 May 18  2009 320.whatis
-r-xr-xr-x  1 root  wheel   620 May 18  2009 999.local


I'm running 10.6.2
hope that helps!

Yes indeed... thanks. Situation is now confirmed.

I just did some deep searching at macosxhints (forums and articles). It doesn't appear as if anyone there has yet noticed that locate's weekly rebuilds have have been stripped from Snowy.

We're the first!!!

Top
#6368 - 12/04/09 02:28 PM Re: shorter maintenance script [Re: Hal Itosis]
Virtual1 Offline


Registered: 08/04/09
Loc: Iowa
I've seen in at least two places where the LOCATE database has basically been abandoned by OS X. It can't be trusted anymore, they're expecting you to use spotlight I suppose. (from terminal eh?)

How is it you trigger the locate database to rebuild?
_________________________
I work for the Department of Redundancy Department

Top
#6376 - 12/04/09 04:19 PM Re: shorter maintenance script [Re: Virtual1]
Hal Itosis Offline


Registered: 09/03/09
Loc: 10.6.8 (build 10K549)
Originally Posted By: Virtual1
I've seen in at least two places where the LOCATE database has basically been abandoned by OS X.

Linky?


Originally Posted By: Virtual1
How is it you trigger the locate database to rebuild?

Simple, i just let launchd work its magic.
[get with the program: i'm using 10.5.8 still. smile ]

Top
#6380 - 12/04/09 07:01 PM Re: shorter maintenance script [Re: Hal Itosis]
Virtual1 Offline


Registered: 08/04/09
Loc: Iowa
I may be mistaken or remembering something wrong here.

cat /etc/defaults/periodic.conf | grep locate
# 310.locate
weekly_locate_enable="YES" # Update locate weekly

The rebuild script is here in 10.5 anyway:
/etc/periodic/weekly/310.locate
_________________________
I work for the Department of Redundancy Department

Top
#6383 - 12/04/09 10:28 PM Re: shorter maintenance script [Re: Virtual1]
artie505 Online


Registered: 08/04/09
> The rebuild script is here in 10.5 anyway:
/etc/periodic/weekly/310.locate


That was more or less Jon's original question; it disappeared in Snow Leopard.
_________________________
The new Great Equalizer is the SEND button.

Top
#6391 - 12/05/09 08:01 AM Re: shorter maintenance script [Re: Virtual1]
Hal Itosis Offline


Registered: 09/03/09
Loc: 10.6.8 (build 10K549)
Originally Posted By: Virtual1
I may be mistaken or remembering something wrong here.

cat /etc/defaults/periodic.conf | grep locate
# 310.locate
weekly_locate_enable="YES" # Update locate weekly

The rebuild script is here in 10.5 anyway:
/etc/periodic/weekly/310.locate

100% correct. (seems the only thing you forgot was to read this thread. wink )

Top
#7300 - 01/05/10 01:56 PM Re: shorter maintenance script [Re: jchuzi]
Hal Itosis Offline


Registered: 09/03/09
Loc: 10.6.8 (build 10K549)
Hey again Jon,
  1. First off...
    Originally Posted By: jchuzi
    Here we go:
    type -a locate{,.updatedb}
    locate is /usr/bin/locate
    /usr/bin/type: line 4: type: locate.updatedb: not found

    New info from macosxhints leads me to think we can get a better fix on this situation. Most likely, that type command failed to show the locate.updatedb executable simply because your shell's $PATH variable doesn't include the /usr/libexec folder.

    To confirm that theory, try this listing (please):

    ls -l /usr/libexec/loc*



  2. Next...
    Originally Posted By: jchuzi
    locate *[0-9][0-9][0-9].locate
    locate: No match.
    (Just as you said, Hal.)

    Did you not get some message as well? (i guess not)

    Reportedly, the weekly locate database building can be easily kicked off by the user. Apple has simply postponed its *initiation* by waiting until the user actually tries to use the locate command...

       locate test

       WARNING: The locate database (/var/db/locate.database) does not exist.
       To create the database, run the following command:

       sudo launchctl load -w /System/Library/LaunchDaemons/com.apple.locate.plist

       Please be aware that the database can take some time to generate; once
       the database has been created, this message will no longer appear.



    ... at least that's what's supposed to happen (the first time), according to a macosxhints forum poster.

    I gather you never saw that message then[?].

    Not sure if you'd like to get that weekly task running or not... but before loading that plist, perhaps you could just read (and post) its content for us?

    To show us what it says and/or determine if it even exists, try this (please):

    defaults read /System/Library/LaunchDaemons/com.apple.locate

Thanks.


Edited by Hal Itosis (01/05/10 02:06 PM)
Edit Reason: fix last command by removing space

Top
#7314 - 01/05/10 04:26 PM Re: shorter maintenance script [Re: Hal Itosis]
jchuzi Offline


Registered: 08/04/09
Loc: New York State
Here we go, Hal:

ls -l /usr/libexec/loc*
-r-xr-xr-x 1 root wheel 50576 May 18 2009 /usr/libexec/locate.bigram
-r-xr-xr-x 1 root wheel 50896 May 18 2009 /usr/libexec/locate.code
-r-xr-xr-x 1 root wheel 2350 May 18 2009 /usr/libexec/locate.concatdb
-r-xr-xr-x 1 root wheel 3150 May 18 2009 /usr/libexec/locate.mklocatedb
-r-xr-xr-x 1 root wheel 3449 May 18 2009 /usr/libexec/locate.updatedb
-rwxr-xr-x 1 root wheel 2091424 Oct 16 08:25 /usr/libexec/locationd

defaults read /System/Library/LaunchDaemons/com.apple.locate
{
AbandonProcessGroup = 1;
Disabled = 1;
KeepAlive = {
PathState = {
"/var/db/locate.database" = 0;
};
};
Label = "com.apple.locate";
LowPriorityIO = 1;
Nice = 5;
ProgramArguments = (
"/usr/libexec/locate.updatedb"
);
StartCalendarInterval = {
Hour = 3;
Minute = 15;
Weekday = 6;
};
}
_________________________
Jon

OS 10.14.2, iMac Retina 5K 27-inch, late 2014, 3.5 GHz Intel Core i5, 1 TB fusion drive, 16 GB RAM, Epson SureColor P600, Photoshop CC, Lightroom CC, MS Office 365

Top
#7321 - 01/05/10 08:15 PM Re: shorter maintenance script [Re: jchuzi]
Hal Itosis Offline


Registered: 09/03/09
Loc: 10.6.8 (build 10K549)
Originally Posted By: jchuzi
ls -l /usr/libexec/loc*
-r-xr-xr-x 1 root wheel 3449 May 18 2009 /usr/libexec/locate.updatedb

Thar she blows.

Had i thought a little harder earlier, i should have realized that your $PATH probably doesn't include /usr/libexec... so therefore "type -a" won't go looking there. [mea maxima culpa]



Originally Posted By: jchuzi
defaults read /System/Library/LaunchDaemons/com.apple.locate
Code:
{
    AbandonProcessGroup = 1;
    Disabled = 1;
    KeepAlive =     {
        PathState =         {
            "/var/db/locate.database" = 0;
        };

And that "Disabled = 1" is the new Apple default. (actually, the whole plist is new... and never existed before. They've simply pulled the "locate" stuff out of the normal weekly routine, and given it its own launchd job).

Thus, as per the pink-colored message above (mentioned in a macosxhints thread), if a Snow Leopard user wants the former weekly update of the locate.database... they must first enable its launchd job, by running:

sudo launchctl load -w /System/Library/LaunchDaemons/com.apple.locate.plist

The -w means "write" ... which will modify that plist, changing "Disabled = 1 to "Disabled = 0" (meaning Enabled).

And now we know the full story methinks.

FWIW, a quick search at Mac OSX Hints main page and Macworld seems to indicate this info has not been handed out to the masses as yet.


Edited by Hal Itosis (01/05/10 08:32 PM)

Top

Moderator:  alternaut, dkmarsh, joemikeb