|
Replies:
11
-
Last Post:
Mar 18, 2008 2:32 PM
by: equach
|
|
|
Posts:
1,013
From:
US
Registered:
3/2/06
|
|
|
|
Snap Upgrade: "syncing" between BEs.
Posted:
Mar 10, 2008 1:00 PM
To: Projects » caiman » discuss
|
|
The following is a proposal on how to manage syncing between BEs
in Snap Upgrade. This was an issue brought up during design review.
Your thoughts/comments are welcome...
Background
----------
There are certain directories and files that we want to be kept
in sync across BEs. An easy example of this is the /var/mail directory.
When rebooting to a different BE, we want that BE to see /var/mail in
the exact state it was in for the previous BE right before reboot.
Some sort of syncing mechanism is needed to accomplish this.
Live Upgrade accomplishes this by actually copying files and
directories from the previously booted BE to the newly booted BE
upon boot up. This potentially requires a large amount of work during
shutdown and bootup. It also, requires maintenance in tracking which
BE was the previously booted BE, etc. Both these are things we would
like to avoid doing in Snap Upgrade.
Requirements
------------
1. Avoid needing "init 6" to boot between BEs (shutdown scripts)
2. Ensure the current BE's namespace is safely protected from
software updates to other BEs.
3. Works for zones.
Proposal
--------
A proposal we have for solving this in Snap Upgrade, is to create
a /var/shared directory which will be its own dataset shared across
all Snap Upgrade BEs. We will move the directories that we want
"synced" under /var/shared and create symlinks from their original
locations. The directories that we are proposing to do this with are:
/var/adm
/var/log
/var/mail
/var/spool
Restrictions/Issues
-------------------
The list of items being synced will no longer be a publicly
modifiable interface. Users will no longer be able to add
arbitrary files or dirs that they want synced.
To keep the current BE's namespace safe, the /var/shared directory
must be mounted read-only for alternate BEs. pkg(5) transactions
against mounted BE alternate roots will require special support.
For this to be supportable with zones, a separate /var/shared
dataset will need to exist for every zone. Upon BE creation,
corresponding zones across BEs share a /var/shared dataset.
|
|
|
Posts:
1,361
From:
US
Registered:
8/5/05
|
|
|
|
Re: Snap Upgrade: "syncing" between BEs.
Posted:
Mar 10, 2008 8:42 PM
in response to: equach
|
|
On Mon, Mar 10, 2008 at 2:00 PM, Ethan Quach <ethan dot quach at sun dot com> wrote:
> Proposal
> --------
> A proposal we have for solving this in Snap Upgrade, is to create
> a /var/shared directory which will be its own dataset shared across
> all Snap Upgrade BEs. We will move the directories that we want
> "synced" under /var/shared and create symlinks from their original
> locations. The directories that we are proposing to do this with are:
I (still) think this is a very good idea. Thanks for listening!
> /var/adm
> /var/log
> /var/mail
> /var/spool
There may be others worth discussing. For details see
http://mail.opensolaris.org/pipermail/caiman-discuss/2007-August/000737.html.
> Restrictions/Issues
> -------------------
> The list of items being synced will no longer be a publicly
> modifiable interface. Users will no longer be able to add
> arbitrary files or dirs that they want synced.
Does it need to be an either/or?
Perhaps keeping the lu style synclist with a DATASET entry would be
best. Merging your list above with the example at
http://docs.sun.com/app/docs/doc/820-0178/6nbufm85f?a=view:
/var/share DATASET
/etc/passwd OVERWRITE
/etc/shadow OVERWRITE
/etc/opasswd OVERWRITE
/etc/oshadow OVERWRITE
/etc/group OVERWRITE
/etc/pwhist OVERWRITE
/etc/default/passwd OVERWRITE
/etc/dfs OVERWRITE
> To keep the current BE's namespace safe, the /var/shared directory
> must be mounted read-only for alternate BEs. pkg(5) transactions
> against mounted BE alternate roots will require special support.
>
> For this to be supportable with zones, a separate /var/shared
> dataset will need to exist for every zone. Upon BE creation,
> corresponding zones across BEs share a /var/shared dataset.
Assuming my suggestion for augmenting rather than replacing synclist
is taken, it would be most desirable to allow each zone to have its
own synclist.
--
Mike Gerdts
http://mgerdts.blogspot.com/
_______________________________________________
caiman-discuss mailing list
caiman-discuss at opensolaris dot org
http://mail.opensolaris.org/mailman/listinfo/caiman-discuss
|
|
|
|
|
Kyle McDonald
KMcDonald@Egenera.COM
|
|
|
|
Re: Snap Upgrade: "syncing" between BEs.
Posted:
Mar 11, 2008 8:40 AM
in response to: mgerdts
|
|
Mike Gerdts wrote:
<pre wrap="">On Mon, Mar 10, 2008 at 2:00 PM, Ethan Quach <ethan dot quach at sun dot com> wrote:
</pre>
<pre wrap=""> Proposal
--------
A proposal we have for solving this in Snap Upgrade, is to create
a /var/shared directory which will be its own dataset shared across
all Snap Upgrade BEs. We will move the directories that we want
"synced" under /var/shared and create symlinks from their original
locations. The directories that we are proposing to do this with are:
</pre>
<pre wrap=""><!---->
</pre>
</blockquote>
Just one quick observation. (appologies if this has been discussed,
this only just appeared in the list I subscribe to.)<br>
<br>
Maybe dropping the 'd' off 'shared' is better? there is already a
/usr/share, and 'share' is found in other places too.<br>
<br>
-Kyle<br>
<br>
<blockquote
cite="mid:65f8f3ad0803102042i634e6bbexe0c0b05567740fb0 at mail dot gmail dot com"
type="cite">
<pre wrap="">I (still) think this is a very good idea. Thanks for listening!
</pre>
<blockquote type="cite">
<pre wrap=""> /var/adm
/var/log
/var/mail
/var/spool
</pre>
</blockquote>
<pre wrap=""><!---->
There may be others worth discussing. For details see
http://mail.opensolaris.org/pipermail/caiman-discuss/2007-August/000737.html.
</pre>
<pre wrap=""> Restrictions/Issues
-------------------
The list of items being synced will no longer be a publicly
modifiable interface. Users will no longer be able to add
arbitrary files or dirs that they want synced.
</pre>
<pre wrap=""><!---->
Does it need to be an either/or?
Perhaps keeping the lu style synclist with a DATASET entry would be
best. Merging your list above with the example at
http://docs.sun.com/app/docs/doc/820-0178/6nbufm85f?a=view:
/var/share DATASET
/etc/passwd OVERWRITE
/etc/shadow OVERWRITE
/etc/opasswd OVERWRITE
/etc/oshadow OVERWRITE
/etc/group OVERWRITE
/etc/pwhist OVERWRITE
/etc/default/passwd OVERWRITE
/etc/dfs OVERWRITE
</pre>
<pre wrap=""> To keep the current BE's namespace safe, the /var/shared directory
must be mounted read-only for alternate BEs. pkg(5) transactions
against mounted BE alternate roots will require special support.
For this to be supportable with zones, a separate /var/shared
dataset will need to exist for every zone. Upon BE creation,
corresponding zones across BEs share a /var/shared dataset.
</pre>
<pre wrap=""><!---->
Assuming my suggestion for augmenting rather than replacing synclist
is taken, it would be most desirable to allow each zone to have its
own synclist.
</pre>
|
|
|
|
|
Posts:
1,361
From:
US
Registered:
8/5/05
|
|
|
|
Re: Snap Upgrade: "syncing" between BEs.
Posted:
Mar 11, 2008 9:54 AM
in response to: Kyle McDonald
|
|
On Tue, Mar 11, 2008 at 10:40 AM, Kyle McDonald <KMcDonald at egenera dot com> wrote:
> Just one quick observation. (appologies if this has been discussed, this
> only just appeared in the list I subscribe to.)
>
> Maybe dropping the 'd' off 'shared' is better? there is already a
> /usr/share, and 'share' is found in other places too.
I agree - otherwise we will have surely new users refering to
/usr/shared if they stumble across /var/shared first.
--
Mike Gerdts
http://mgerdts.blogspot.com/
_______________________________________________
caiman-discuss mailing list
caiman-discuss at opensolaris dot org
http://mail.opensolaris.org/mailman/listinfo/caiman-discuss
|
|
|
|
|
Posts:
1,013
From:
US
Registered:
3/2/06
|
|
|
|
Re: Snap Upgrade: "syncing" between BEs.
Posted:
Mar 11, 2008 4:34 PM
in response to: mgerdts
|
|
Mike Gerdts wrote:
> On Mon, Mar 10, 2008 at 2:00 PM, Ethan Quach <ethan dot quach at sun dot com> wrote:
>> Proposal
>> --------
>> A proposal we have for solving this in Snap Upgrade, is to create
>> a /var/shared directory which will be its own dataset shared across
>> all Snap Upgrade BEs. We will move the directories that we want
>> "synced" under /var/shared and create symlinks from their original
>> locations. The directories that we are proposing to do this with are:
>
> I (still) think this is a very good idea. Thanks for listening!
>
>> /var/adm
>> /var/log
>> /var/mail
>> /var/spool
>
> There may be others worth discussing. For details see
> http://mail.opensolaris.org/pipermail/caiman-discuss/2007-August/000737.html.
I came to the above list by looking at what should be required
to be shared across BEs vs. what could just be optional. What we need
to be careful in considering is not only the format, but also the
existence of what were sharing across BEs and software version updates.
From looking again at what's listed in that thread, perhaps a few
more that could be considered for sharing are:
audit cron idmap inet news preserve saf
>
>
>> Restrictions/Issues
>> -------------------
>> The list of items being synced will no longer be a publicly
>> modifiable interface. Users will no longer be able to add
>> arbitrary files or dirs that they want synced.
>
> Does it need to be an either/or?
>
> Perhaps keeping the lu style synclist with a DATASET entry would be
> best. Merging your list above with the example at
> http://docs.sun.com/app/docs/doc/820-0178/6nbufm85f?a=view:
>
> /var/share DATASET
> /etc/passwd OVERWRITE
> /etc/shadow OVERWRITE
> /etc/opasswd OVERWRITE
> /etc/oshadow OVERWRITE
> /etc/group OVERWRITE
> /etc/pwhist OVERWRITE
> /etc/default/passwd OVERWRITE
> /etc/dfs OVERWRITE
Maintaining this synclist means supporting some form of shutdown/startup
processing, which is what we're trying to avoid. The idea is to
intentionally take this away, and stop facilitating the ability to
specify arbitrary system files and directories to sync. What type of
things do you see an admin wanting to add?
These /etc/* files from the lu synclist are things we're proposing to
drop from being sync'ed.
_______________________________________________
caiman-discuss mailing list
caiman-discuss at opensolaris dot org
http://mail.opensolaris.org/mailman/listinfo/caiman-discuss
|
|
|
|
|
Posts:
72
From:
Registered:
5/30/06
|
|
|
|
Re: Snap Upgrade: "syncing" between BEs.
Posted:
Mar 11, 2008 10:27 PM
in response to: equach
To: Projects » caiman » discuss
|
|
> Proposal
> --------
> A proposal we have for solving this in Snap Upgrade, is to create
> a /var/shared directory which will be its own dataset shared across
> all Snap Upgrade BEs. We will move the directories that we want
> "synced" under /var/shared and create symlinks from their original
> locations. The directories that we are proposing to do this with are:
I agree with the comment about using /var/share instead
> /var/adm
> /var/log
> /var/mail
> /var/spool
These should also be strongly considered:
/var/crash
/var/core (a common place to put core files using coreadm)
Paul
|
|
|
|
|
Posts:
1,013
From:
US
Registered:
3/2/06
|
|
|
|
Re: Snap Upgrade: "syncing" between BEs.
Posted:
Mar 11, 2008 11:59 PM
in response to: psa
|
|
Paul Armstrong wrote:
>> /var/adm
>> /var/log
>> /var/mail
>> /var/spool
>>
>
> These should also be strongly considered:
> /var/crash
> /var/core (a common place to put core files using coreadm)
>
Perhaps, although these are a bit interesting because they are a product
of a configuration rather than a registered directory delivered by a pkg.
We could just change the default dumpadm and coreadm configurations
to address these, yes?
-ethan
_______________________________________________
caiman-discuss mailing list
caiman-discuss at opensolaris dot org
http://mail.opensolaris.org/mailman/listinfo/caiman-discuss
|
|
|
|
|
Kyle McDonald
KMcDonald@Egenera.COM
|
|
|
|
Re: Snap Upgrade: "syncing" between BEs.
Posted:
Mar 12, 2008 5:57 AM
in response to: equach
|
|
Ethan Quach wrote:
<pre wrap="">
Paul Armstrong wrote:
</pre>
<pre wrap="">/var/adm
/var/log
/var/mail
/var/spool
</pre>
<pre wrap="">These should also be strongly considered:
/var/crash
/var/core (a common place to put core files using coreadm)
</pre>
<pre wrap=""><!---->
Perhaps, although these are a bit interesting because they are a product
of a configuration rather than a registered directory delivered by a pkg.
We could just change the default dumpadm and coreadm configurations
to address these, yes?
</pre>
You mean reconfigure them to always go to /var/share/crash or
/var/share/core by default and then not need softlinks?
Or do you mean somehow reconfiguring them at the time the admin
switches which is the active BE?
Or do you mean something else entirely?
The first seems acceptable to me. I don't mind softlinks, but if this
would let use avoid them that seems better even.
-Kyle
<pre wrap="">
-ethan
_______________________________________________
caiman-discuss mailing list
caiman-discuss at opensolaris dot org
http://mail.opensolaris.org/mailman/listinfo/caiman-discuss
</pre>
|
|
|
|
|
Posts:
1,013
From:
US
Registered:
3/2/06
|
|
|
|
Re: Snap Upgrade: "syncing" between BEs.
Posted:
Mar 12, 2008 9:12 AM
in response to: Kyle McDonald
|
|
Kyle McDonald wrote:
> Ethan Quach wrote:
>> Paul Armstrong wrote:
>>
>>>> /var/adm
>>>> /var/log
>>>> /var/mail
>>>> /var/spool
>>>>
>>>>
>>> These should also be strongly considered:
>>> /var/crash
>>> /var/core (a common place to put core files using coreadm)
>>>
>>>
>>
>> Perhaps, although these are a bit interesting because they are a product
>> of a configuration rather than a registered directory delivered by a pkg.
>> We could just change the default dumpadm and coreadm configurations
>> to address these, yes?
>>
>>
> You mean reconfigure them to always go to /var/share/crash or
> /var/share/core by default and then not need softlinks?
> Or do you mean somehow reconfiguring them at the time the admin switches
> which is the active BE?
>
> Or do you mean something else entirely?
>
> The first seems acceptable to me. I don't mind softlinks, but if this
> would let use avoid them that seems better even.
I meant the former, minus the "always". Core and crash locations are
admin chosen configurations after all.
-ethan
_______________________________________________
caiman-discuss mailing list
caiman-discuss at opensolaris dot org
http://mail.opensolaris.org/mailman/listinfo/caiman-discuss
|
|
|
|
|
Posts:
1,361
From:
US
Registered:
8/5/05
|
|
|
|
Re: Snap Upgrade: "syncing" between BEs.
Posted:
Mar 12, 2008 5:19 AM
in response to: psa
|
|
On Wed, Mar 12, 2008 at 12:27 AM, Paul Armstrong <opensolaris at otoh dot org> wrote:
> These should also be strongly considered:
> /var/crash
> /var/core (a common place to put core files using coreadm)
Using the name "core" as a directory name makes it so that anything
with a cwd of /var (or /var/share) cannot do per-process core dumps.
I currently configure systems so that they *try* to core dump to
/var/cores/core.<some complex string>. Under most circumstances
/var/cores does not exist - else it would be easy for users to
accidentally fill up the file system in such a way that only the
system administrator can fix it. This has the nice side effect that I
get a syslog message saying that the system could not write a core
file to /var/cores - in other words I get a message indicating that
programs are crashing. This shortens the time to answer things like
"which system administrator keeps killing processes on my server?"
I think, and Sun's best practices agree, that having such a directory
for catching all core dumps is problematic because of the space
consumption issue. I am happy to have people do it if it brings more
value than harm to them, but I don' t think it is right as a default.
I agree that /var/crash belongs there. And several other things
probably do too. See
http://mail.opensolaris.org/pipermail/caiman-discuss/2007-August/000737.html
for a somewhat comprehensive list of /var directories that are not
controlled by the package database and as such should be considered as
to whether you really want a copy that is private to a boot
environment.
--
Mike Gerdts
http://mgerdts.blogspot.com/
_______________________________________________
caiman-discuss mailing list
caiman-discuss at opensolaris dot org
http://mail.opensolaris.org/mailman/listinfo/caiman-discuss
|
|
|
|
|
Posts:
1,013
From:
US
Registered:
3/2/06
|
|
|
|
Re: Snap Upgrade: "syncing" between BEs.
Posted:
Mar 18, 2008 2:32 PM
in response to: mgerdts
|
|
Mike Gerdts wrote:
>> Just one quick observation. (appologies if this has been
>> discussed, this only just appeared in the list I subscribe to.)
>>
>> Maybe dropping the 'd' off 'shared' is better? there is already a
>> /usr/share, and 'share' is found in other places too.
>
> I agree - otherwise we will have surely new users refering to
> /usr/shared if they stumble across /var/shared first.
>
This sounds better to me too.
>
> I agree that /var/crash belongs there. And several other things
> probably do too. See
> http://mail.opensolaris.org/pipermail/caiman-discuss/2007-August/000737.html
> for a somewhat comprehensive list of /var directories that are not
> controlled by the package database and as such should be considered as
> to whether you really want a copy that is private to a boot
> environment.
We'll need to take a closer look at each one of these to see if
/var/share is appropriate, and if so, we'll make the move. For now, I'd
like to push forward on the idea of /var/share. I'll be sending out a
proposal to capture this. Thoughts and comments on that will be much
welcomed.
thanks,
-ethan
_______________________________________________
caiman-discuss mailing list
caiman-discuss at opensolaris dot org
http://mail.opensolaris.org/mailman/listinfo/caiman-discuss
|
|
|
|
|
Posts:
813
From:
US
Registered:
3/9/05
|
|
|
|
Re: Snap Upgrade: "syncing" between BEs.
Posted:
Mar 12, 2008 9:04 AM
in response to: psa
|
|
Paul Armstrong wrote:
>> Proposal
>> --------
>> A proposal we have for solving this in Snap Upgrade, is to create
>> a /var/shared directory which will be its own dataset shared across
>> all Snap Upgrade BEs. We will move the directories that we want
>> "synced" under /var/shared and create symlinks from their original
>> locations. The directories that we are proposing to do this with are:
>>
>
> I agree with the comment about using /var/share instead
>
>
>> /var/adm
>> /var/log
>> /var/mail
>> /var/spool
>>
>
> These should also be strongly considered:
> /var/crash
> /var/core (a common place to put core files using coreadm)
>
I'm probably missing something here but I don't yet see why we'd want
/var/crash in a shared area. With this separate you'll know which BE
caused the crash however if this is a shared area where all BEs put their
crash dumps you'll have a difficult time keeping track of which BE
caused which crash.
-evan
_______________________________________________
caiman-discuss mailing list
caiman-discuss at opensolaris dot org
http://mail.opensolaris.org/mailman/listinfo/caiman-discuss
|
|
|
|
|
