Patchwork Use a tag for SeaBIOS stable checkouts.

login
register
about
Submitter Peter Stuge
Date 2011-01-23 23:35:05
Message ID <20110123233505.32567.qmail@stuge.se>
Download mbox | patch
Permalink /patch/2551/
State Superseded
Headers show

Comments

Peter Stuge - 2011-01-23 23:35:05
Stefan Reinauer wrote:
> Not sure what the right fix is for this.

We'll find one, but need to decide on how to handle the situation
where there have been changes made in the repo. Both commited ones
and uncommitted ones (like what make patch currently produces).


> I neither want to check the changes I make to the config file in to
> the local repository

Can we please not use perl for this? sed can do it just fine.

Can I ask why you don't want to create a commit for the changes?


> nor do I want to undo/redo the changes for every build,

Well, but every build can (outside abuild) switch to a different
branch. And we need to handle that. (Ie. a user first building with
master, then building with stable.)


> nor do I want to re-checkout for every target.

Why not? Note we are talking git checkout now, not git clone.


> Is this possible with git?

Yes, it can do whatever we decide on, but so far we haven't really
considered all cases, so what we have right now doesn't work.


Kevin O'Connor wrote:
> The following makes it less of a problem for me, but I'm not sure it's
> the right way either.
..
> +++ b/payloads/external/SeaBIOS/Makefile.inc
> -TAG-$(CONFIG_SEABIOS_MASTER)=master
> +TAG-$(CONFIG_SEABIOS_MASTER)=origin/master
>  TAG-$(CONFIG_SEABIOS_STABLE)=rel-0.6.1.3
..
>  checkout:
>         echo "Checking out SeaBIOS $(TAG-y)"
> -       test -d seabios && ( cd seabios; git pull ) || \
> +       test -d seabios && ( cd seabios; git fetch ) || \

I agree that git fetch should be here, certainly not git pull.

But we need to consider what we want to happen if someone has made
local changes in this repository. And we also need to consider *our*
local changes that are done in this repository. The fact that they
are not so easily distinguishable is a problem.


> -       cd seabios; git checkout $(TAG-y)
> +       cd seabios; git checkout -m $(TAG-y)

I don't really like trying to do a merge of local changes here. It
will quickly conflict with later development from upstream.


We need to weigh two objectives against each other:

1. someone has local changes in payloads/external/SeaBIOS/seabios
that we don't want to erase

2. we want make to use the particular version of SeaBIOS specified
by the user or by abuild during config


> The best way is probably to never checkout a branch - only ever pull
> a tag.  Then the build can detect if the tag ever changes and reapply
> the patch.

I think merging is always the wrong thing to do, because it will make
a mess of the repo and what the user has configured is exclusively
one of the tags we offer, so that is what they should always get.

What I'm saying is that I think objective 2. above takes priority
over objective 1. and then we can not really have local changes in
the seabios repo. That means *any* local changes, theirs or ours.

The .patched thingy is sorta an ugly trick too. As I see it, what we
want is to build a certain way, without having to change any files,
so I think we should pass CONFIG_ on the command line. Patches
attached.


//Peter
From be44530690aabfe4c66351f842bf073e2728ce02 Mon Sep 17 00:00:00 2001
From: Peter Stuge <peter@stuge.se>
Date: Mon, 24 Jan 2011 00:15:18 +0100
Subject: [PATCH 1/2] Allow initial COMMONCFLAGS to be set from the make command line

---
 Makefile |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
Stefan Reinauer - 2011-01-25 01:37:03
* Peter Stuge <peter@stuge.se> [110124 00:35]:
> > I neither want to check the changes I make to the config file in to
> > the local repository
> 
> Can we please not use perl for this? sed can do it just fine.
 
Sure. Not all sed versions support in-place changes though, so the code
will have to handle that. Just send the patch, I'll ack.

> Can I ask why you don't want to create a commit for the changes?

It needs more of git setup in order to use the system (such as the user
name and email), so I am hesitating to put this constraint on the users
for something that should not require a commit in the first place. It's
only a configuration change.
 
> > nor do I want to undo/redo the changes for every build,
> 
> Well, but every build can (outside abuild) switch to a different
> branch. And we need to handle that. (Ie. a user first building with
> master, then building with stable.)

I believe that my original code did that, at least it worked in my
tests.

> > nor do I want to re-checkout for every target.
> 
> Why not? Note we are talking git checkout now, not git clone.

No, actually I was talking about git clone. Sorry, I am not quite used
to all the git nomenclature yet. In fact git checkout simply failed
for me with varying errors.

> > +TAG-$(CONFIG_SEABIOS_MASTER)=origin/master
> >  TAG-$(CONFIG_SEABIOS_STABLE)=rel-0.6.1.3
> ..
> >  checkout:
> >         echo "Checking out SeaBIOS $(TAG-y)"
> > -       test -d seabios && ( cd seabios; git pull ) || \
> > +       test -d seabios && ( cd seabios; git fetch ) || \
> 
> I agree that git fetch should be here, certainly not git pull.

A fetch alone does not seem to make much sense. It will not update the
tree when working with the master tag, as far as I can tell. The idea of
master is that you will always get the latest and greatest SeaBIOS, so 
the tree should be updated accordingly, eh, fetched and merged.

> But we need to consider what we want to happen if someone has made
> local changes in this repository. And we also need to consider *our*
> local changes that are done in this repository. The fact that they
> are not so easily distinguishable is a problem.

Let's start with the simple case of us just building a plain image from
the SeaBIOS git repository without the user changing stuff on top of
that. Remember, this is just the "simple path". If you want to develop
SeaBIOS and change it, you can always specify it as external ELF payload
instead of using this automatism.

> > -       cd seabios; git checkout $(TAG-y)
> > +       cd seabios; git checkout -m $(TAG-y)
> 
> I don't really like trying to do a merge of local changes here. It
> will quickly conflict with later development from upstream.

Well, unless SeaBIOS switches over to a non-checked-in, auto generated
config.h we will probably have to do this. Do we want to switch SeaBIOS
over to Kconfig? That would certainly solve the issue, and allow to
specify a default config, as well as user or coreboot specific changes
to the configuration.

> We need to weigh two objectives against each other:
> 
> 1. someone has local changes in payloads/external/SeaBIOS/seabios
> that we don't want to erase

I think that's out of focus, as described above. However, we need to
configure SeaBIOS to actually work on top of coreboot, and that requires
automatically doing "local changes" due to the way the code works currently.

> I think merging is always the wrong thing to do, because it will make
> a mess of the repo and what the user has configured is exclusively
> one of the tags we offer, so that is what they should always get.

I think it's quite incredible that we can not update the version of
SeaBIOS automatically because we enabled it to build with coreboot and
serial console support. We need to fix this constraint.

> Subject: [PATCH 1/2] Allow initial COMMONCFLAGS to be set from the make command line

The patch will probably allow CFLAGS like -O2 be overwritten by -Os?

I think COMMONCFLAGS is not something the user should set. The commonly
used notion would suggest the user set CFLAGS instead.
 
>  Makefile |    2 +-
>  1 files changed, 1 insertions(+), 1 deletions(-)
> 
> diff --git a/Makefile b/Makefile
> index 384bf79..ccdf96a 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -27,7 +27,7 @@ cc-option = $(shell if test -z "`$(1) $(2) -S -o /dev/null -xc \
>                /dev/null 2>&1`"; then echo "$(2)"; else echo "$(3)"; fi ;)
>  
>  # Default compiler flags
> -COMMONCFLAGS = -Os -MD -Wall -Wno-strict-aliasing -Wold-style-definition \
> +COMMONCFLAGS += -Os -MD -Wall -Wno-strict-aliasing -Wold-style-definition \
>                 $(call cc-option,$(CC),-Wtype-limits,) \
>                 -m32 -march=i386 -mregparm=3 -mpreferred-stack-boundary=2 \
>                 -mrtd -minline-all-stringops \
 

> From: Peter Stuge <peter@stuge.se>
> Date: Mon, 24 Jan 2011 00:20:18 +0100
> Subject: [PATCH 2/2] Allow override of CONFIG_COREBOOT CONFIG_DEBUG_SERIAL CONFIG_VGAHOOKS
> 
> This allows the coreboot build system control over these parameters
> without having to modify src/config.h.
> ---
>  src/config.h |    8 ++++++++
>  1 files changed, 8 insertions(+), 0 deletions(-)
> 
> diff --git a/src/config.h b/src/config.h
> index db3e578..f0cba10 100644
> --- a/src/config.h
> +++ b/src/config.h
> @@ -13,12 +13,18 @@
>  #define CONFIG_APPNAME4 "BXPC"
>  
>  // Configure as a coreboot payload.
> +#ifndef CONFIG_COREBOOT
>  #define CONFIG_COREBOOT 0
> +#endif
>  
>  // Control how verbose debug output is.
> +#ifndef CONFIG_DEBUG_LEVEL
>  #define CONFIG_DEBUG_LEVEL 1
> +#endif
>  // Send debugging information to serial port
> +#ifndef CONFIG_DEBUG_SERIAL
>  #define CONFIG_DEBUG_SERIAL 0
> +#endif
>  // Screen writes are also sent to debug ports.
>  #define CONFIG_SCREEN_AND_DEBUG 1
>  
> @@ -120,7 +126,9 @@
>  // Support generation of ACPI tables (for emulators)
>  #define CONFIG_ACPI 1
>  // Support bios callbacks specific to via vgabios.
> +#ifndef CONFIG_VGAHOOKS
>  #define CONFIG_VGAHOOKS 0
> +#endif
>  // Support S3 resume handler.
>  #define CONFIG_S3_RESUME 1
>  // Run the vga rom during S3 resume.
>

Should other flags be exported as well? All of them? 

I like the approach, and it is the right thing to do, not changing the
code. But it also adds "logic" to the config file, which is a bit eerie.

> Index: payloads/external/SeaBIOS/Makefile.inc
> ===================================================================
> --- payloads/external/SeaBIOS/Makefile.inc	(revision 6292)
> +++ payloads/external/SeaBIOS/Makefile.inc	(working copy)
> @@ -5,20 +5,12 @@
>  
>  all: seabios
>  
> -seabios: patch
> -	cd seabios; $(MAKE) CC="$(CC)" LD="$(LD)"
> +seabios: checkout
> +	cd seabios; $(MAKE) CC="$(CC)" LD="$(LD)" COMMONCFLAGS="-DCONFIG_COREBOOT=1 -DCONFIG_DEBUG_SERIAL=1 -DCONFIG_VGAHOOKS=1"
>  
> -patch: checkout
> -	test -r seabios/.patched || \
> -	perl -pi -e "s,#define CONFIG_COREBOOT 0,#define CONFIG_COREBOOT 1,;" \
> -		 -e "s,#define CONFIG_DEBUG_SERIAL 0,#define CONFIG_DEBUG_SERIAL 1,;" \
> -		 -e "s,#define CONFIG_VGAHOOKS 0,#define CONFIG_VGAHOOKS 1,;" \
> -		 seabios/src/config.h
> -	touch seabios/.patched
> -
>  checkout:
>  	echo "Checking out SeaBIOS $(TAG-y)"
> -	test -d seabios && ( cd seabios; git pull ) || \
> +	test -d seabios && ( cd seabios; git fetch ) || \

will this actually make sure I am running the latest version when
selecting master?

If so, I think we should go with this until we have a better solution.
It will require the above config.h change to be checked into a new
stable release first though, and our stable release version in
payloads/external/SeaBIOS/Makefile.inc should be pushed to that version
then.

Stefan
Kevin O'Connor - 2011-01-25 02:58:03
On Tue, Jan 25, 2011 at 02:37:03AM +0100, Stefan Reinauer wrote:
> * Peter Stuge <peter@stuge.se> [110124 00:35]:
> > > +TAG-$(CONFIG_SEABIOS_MASTER)=origin/master
> > >  TAG-$(CONFIG_SEABIOS_STABLE)=rel-0.6.1.3
> > ..
> > >  checkout:
> > >         echo "Checking out SeaBIOS $(TAG-y)"
> > > -       test -d seabios && ( cd seabios; git pull ) || \
> > > +       test -d seabios && ( cd seabios; git fetch ) || \
> > 
> > I agree that git fetch should be here, certainly not git pull.
> 
> A fetch alone does not seem to make much sense. It will not update the
> tree when working with the master tag, as far as I can tell. The idea of
> master is that you will always get the latest and greatest SeaBIOS, so 
> the tree should be updated accordingly, eh, fetched and merged.

The fetch should be okay, because the patch also changes the tag from
"master" to "origin/master".

The real fix is to move SeaBIOS to Kconfig.

It's okay if you wish to backout my previous patch.  I apologize for
not catching the build failure on subsequent builds.

-Kevin
Peter Stuge - 2011-01-25 15:18:42
Stefan Reinauer wrote:
> > Can I ask why you don't want to create a commit for the changes?
> 
> It needs more of git setup in order to use the system (such as the user
> name and email),

That's a point, though it doesn't matter significantly because the
commit would not really be sent to anyone else. (git can still commit
without explicit config, it'll just default to $USER@$(hostname -f)
for the email.)

Require config I agree we must not. But having a commit for the
changes in the git repo is I think something we could consider.
However..


> so I am hesitating to put this constraint on the users for
> something that should not require a commit in the first place.
> It's only a configuration change.

The problem is that we want to do a configuration change, but that
configuration file is being managed by git as part of the sources.

So after the configuration file has change, git expects to be told
what to do with the change, since the file *is* part of the repo.


> > > nor do I want to undo/redo the changes for every build,
> > 
> > Well, but every build can (outside abuild) switch to a different
> > branch. And we need to handle that. (Ie. a user first building with
> > master, then building with stable.)
> 
> I believe that my original code did that, at least it worked in my
> tests.

A file in the working copy being modified blocks most operations in
git because it can't know the intent of the changes, so I'd be
surprised if git pull worked while config.h had uncommitted changes.


> > > nor do I want to re-checkout for every target.
> > 
> > Why not? Note we are talking git checkout now, not git clone.
> 
> No, actually I was talking about git clone. Sorry, I am not quite used
> to all the git nomenclature yet.

No problem. And I agree with this, clone should always just be done
once. And git helps with that.


> In fact git checkout simply failed for me with varying errors.

I can see a couple of reasons why it would. Git doesn't allow a
checkout if it can't tell that the current state of the working copy
will be "easily" reproducable. That of course includes uncommitted
changes, but it would also include having no uncommitted changes, if
the current worktree state is not on a local branch, such as would be
the case after a cycle of checkout origin/master + commit config.h
changes.


> > >  checkout:
> > >         echo "Checking out SeaBIOS $(TAG-y)"
> > > -       test -d seabios && ( cd seabios; git pull ) || \
> > > +       test -d seabios && ( cd seabios; git fetch ) || \
> > 
> > I agree that git fetch should be here, certainly not git pull.
> 
> A fetch alone does not seem to make much sense.

Note "here". The checkout comes just afterwards.


> It will not update the tree when working with the master tag, as
> far as I can tell.

Correct, git fetch will *never* update the worktree. It will however
download new changes from upstream, and they will be available in the
branch origin/master that is created automatically whenever cloning a
repo.


> The idea of master is that you will always get the latest and
> greatest SeaBIOS, so the tree should be updated accordingly, eh,
> fetched and merged.

Not neccessarily merged (git merge or other) because it would mean
that what is in the local work tree has departed from what is
upstream, but more importantly because it makes it awkward to jump
around in the repository (switch between master and stable e.g.)
because what is actually configuration is handled as if it were
source code.


> Let's start with the simple case of us just building a plain image from
> the SeaBIOS git repository without the user changing stuff on top of
> that. Remember, this is just the "simple path". If you want to develop
> SeaBIOS and change it, you can always specify it as external ELF payload
> instead of using this automatism.

Great. I agree with this very much, because it allows us to solve the
particular use case, and then we can also tell people what to do and
not do to keep it working.


> > > -       cd seabios; git checkout $(TAG-y)
> > > +       cd seabios; git checkout -m $(TAG-y)
> > 
> > I don't really like trying to do a merge of local changes here. It
> > will quickly conflict with later development from upstream.
> 
> Well, unless SeaBIOS switches over to a non-checked-in, auto generated
> config.h we will probably have to do this. Do we want to switch SeaBIOS
> over to Kconfig? That would certainly solve the issue, and allow to
> specify a default config, as well as user or coreboot specific changes
> to the configuration.

Actually Marc Bertens is working on this as we speak over on the
SeaBIOS mailing list, but it would not be neccessary to go all the
way there to solve the problem.

Renaming config.h to config.h.in and having a configuration step be
part of the build (it could be called from Makefile, so still only
need to run make for default settings) that generates config.h would
work. In that case, Git would be told to always ignore config.h, and
it would never matter if it was changed by coreboot build.

It essentially does the same as my COMMONCFLAGS idea, just stores the
actual settings in a file, which I think has clear benefits.


> I think that's out of focus, as described above. However, we need to
> configure SeaBIOS to actually work on top of coreboot, and that requires
> automatically doing "local changes" due to the way the code works currently.

Right. We will need to change SeaBIOS, but not very much. I think my
patches might be the minimal solution, but nicer and only slightly
larger would be to have a config.h.in/config.h dance, where the major
benefit is that actual used configuration gets stored on the
filesystem and not just in the environment.


> > I think merging is always the wrong thing to do, because it will make
> > a mess of the repo and what the user has configured is exclusively
> > one of the tags we offer, so that is what they should always get.
> 
> I think it's quite incredible that we can not update the version of
> SeaBIOS automatically because we enabled it to build with coreboot
> and serial console support. We need to fix this constraint.

See above for an attempt at explanation. The issue is that we have
made changes to the source code, without telling git how it should
treat those changes. We can give git pretty much any instruction we
want, but we do need to give it instruction. Does that make sense? :)


> > Subject: [PATCH 1/2] Allow initial COMMONCFLAGS to be set from the make command line
> 
> The patch will probably allow CFLAGS like -O2 be overwritten by -Os?
> 
> I think COMMONCFLAGS is not something the user should set. The commonly
> used notion would suggest the user set CFLAGS instead.

I don't care at all about the actual names of variables used, the
point was just to suggest using environment for configuration, since
it should not really be part of the (version controlled) source code.
However, I think a configuration pass would be far superior.


> > Subject: [PATCH 2/2] Allow override of CONFIG_COREBOOT CONFIG_DEBUG_SERIAL CONFIG_VGAHOOKS
> 
> Should other flags be exported as well? All of them? 

Maybe. I just did the ones that were touched by coreboot. I thought
that others could be added as the need comes up.


> I like the approach, and it is the right thing to do, not changing the
> code. But it also adds "logic" to the config file, which is a bit eerie.

Well, yes, and config becomes volatile, which is my main objection.


> >  checkout:
> >  	echo "Checking out SeaBIOS $(TAG-y)"
> > -	test -d seabios && ( cd seabios; git pull ) || \
> > +	test -d seabios && ( cd seabios; git fetch ) || \
> 
> will this actually make sure I am running the latest version when
> selecting master?

No, but the git checkout command that you cut away will. :)


> If so, I think we should go with this until we have a better solution.
> It will require the above config.h change to be checked into a new
> stable release first though, and our stable release version in
> payloads/external/SeaBIOS/Makefile.inc should be pushed to that
> version then.

Right, this problem stretches across both SeaBIOS and coreboot, so we
should make sure the solution is one we actually like. :)


Kevin O'Connor wrote:
> > > > +       test -d seabios && ( cd seabios; git fetch ) || \
> > > 
> > > I agree that git fetch should be here, certainly not git pull.
> > 
> > A fetch alone does not seem to make much sense. It will not update the
> > tree when working with the master tag, as far as I can tell. The idea of
> > master is that you will always get the latest and greatest SeaBIOS, so 
> > the tree should be updated accordingly, eh, fetched and merged.
> 
> The fetch should be okay, because the patch also changes the tag from
> "master" to "origin/master".

Ah, yeah, git fetch could certainly become git fetch origin to become
perfectly clear. It will fetch all branches, both stable and master.


> The real fix is to move SeaBIOS to Kconfig.

Or if that is too big a step maybe config.h.in would be a useful
first step.


//Peter

Patch

diff --git a/Makefile b/Makefile
index 384bf79..ccdf96a 100644
--- a/Makefile
+++ b/Makefile
@@ -27,7 +27,7 @@  cc-option = $(shell if test -z "`$(1) $(2) -S -o /dev/null -xc \
               /dev/null 2>&1`"; then echo "$(2)"; else echo "$(3)"; fi ;)
 
 # Default compiler flags
-COMMONCFLAGS = -Os -MD -Wall -Wno-strict-aliasing -Wold-style-definition \
+COMMONCFLAGS += -Os -MD -Wall -Wno-strict-aliasing -Wold-style-definition \
                $(call cc-option,$(CC),-Wtype-limits,) \
                -m32 -march=i386 -mregparm=3 -mpreferred-stack-boundary=2 \
                -mrtd -minline-all-stringops \