Patchwork AMD MMCONF Support

login
register
about
Submitter Arne Georg Gleditsch
Date 2010-09-13 12:44:23
Message ID <87tyltrexk.fsf@taniquetil.gledits.ch>
Download mbox | patch
Permalink /patch/1922/
State Accepted
Headers show

Comments

Arne Georg Gleditsch - 2010-09-13 12:44:23
Arne Georg Gleditsch <arne.gleditsch@numascale.com> writes:
> With this change, I can re-insert the RES_PCI_IO wihtout ill effects.

Move initialization of MMCONF BAR to cache_as_ram setup phase, in order
to make sure MMCONF is set up before use.  Otherwise, PCI config
accesses run before init_cpus() will be lost if MMCONF is enabled
(unless explicitly done as port-based accesses).

This obsoletes removal of RES_PCI_IO, PCI_ADDR(0, 1, 0, 0x78) in
mcp55_early_setup, so reinsert.

Signed-off-by: Arne Georg Gleditsch <arne.gleditsch@numascale.com>
Myles Watson - 2010-09-13 12:52:23
> Arne Georg Gleditsch <arne.gleditsch@numascale.com> writes:
> > With this change, I can re-insert the RES_PCI_IO wihtout ill effects.
> 
> Move initialization of MMCONF BAR to cache_as_ram setup phase, in order
> to make sure MMCONF is set up before use.  Otherwise, PCI config
> accesses run before init_cpus() will be lost if MMCONF is enabled
> (unless explicitly done as port-based accesses).
> 
> This obsoletes removal of RES_PCI_IO, PCI_ADDR(0, 1, 0, 0x78) in
> mcp55_early_setup, so reinsert.
> 
> Signed-off-by: Arne Georg Gleditsch <arne.gleditsch@numascale.com>

Acked-by: Myles Watson <mylesgw@gmail.com>

Thanks for tracking it down!

Myles
Myles Watson - 2010-09-13 15:15:47
On Mon, Sep 13, 2010 at 6:52 AM, Myles Watson <mylesgw@gmail.com> wrote:
>
>> Arne Georg Gleditsch <arne.gleditsch@numascale.com> writes:
>> > With this change, I can re-insert the RES_PCI_IO wihtout ill effects.
>>
>> Move initialization of MMCONF BAR to cache_as_ram setup phase, in order
>> to make sure MMCONF is set up before use.  Otherwise, PCI config
>> accesses run before init_cpus() will be lost if MMCONF is enabled
>> (unless explicitly done as port-based accesses).
>>
>> This obsoletes removal of RES_PCI_IO, PCI_ADDR(0, 1, 0, 0x78) in
>> mcp55_early_setup, so reinsert.
>>
>> Signed-off-by: Arne Georg Gleditsch <arne.gleditsch@numascale.com>
>
> Acked-by: Myles Watson <mylesgw@gmail.com>
Rev 5810.

Arne:
Could you help me understand MMCONF_SUPPORT and
MMCONF_SUPPORT_DEFAULT?  It looks like the area gets reserved for
MMCONF_SUPPORT, even if MMCONF_SUPPORT_DEFAULT isn't selected.

Thanks,
Myles

Thanks,
Myles
Arne Georg Gleditsch - 2010-09-13 15:49:28
Myles Watson <mylesgw@gmail.com> writes:
> Rev 5810.

Thanks!

> Arne:
> Could you help me understand MMCONF_SUPPORT and
> MMCONF_SUPPORT_DEFAULT?  It looks like the area gets reserved for
> MMCONF_SUPPORT, even if MMCONF_SUPPORT_DEFAULT isn't selected.

My understanding of the code as it was before I started messing with it,
is that MMCONF_SUPPORT is intended to indicate whether the facility is
available at all, and MMCONF_SUPPORT_DEFAULT toggles the actual use of
it (by Coreboot).  Both src/northbridge/amd/amdfam10/northbridge.c and
src/northbridge/intel/i945/northbridge.c contains code like this:

  #if CONFIG_MMCONF_SUPPORT_DEFAULT
          .ops_pci_bus      = &pci_ops_mmconf,
  #else
          .ops_pci_bus      = &pci_cf8_conf1,
  #endif

which, alongside romcc_io.h's 

  #if CONFIG_MMCONF_SUPPORT_DEFAULT
          return pci_mmio_read_config32(dev, where);
  #else
          return pci_io_read_config32(dev, where);
  #endif

is more or less the only use for MMCONF_SUPPORT_DEFAULT as far as I can
see.  I'll leave it to others to determine if this actually makes sense.

(Having SUPPORT without SUPPORT_DEFAULT gives you an MCFG area that you
can communicate to the OS (at least potentially), as well as the option
of explicitly doing individual config accesses via mmio instead of
ports.  I don't think anyone is doing either of these today.)
Myles Watson - 2010-09-13 16:44:40
>> Arne:
>> Could you help me understand MMCONF_SUPPORT and
>> MMCONF_SUPPORT_DEFAULT?  It looks like the area gets reserved for
>> MMCONF_SUPPORT, even if MMCONF_SUPPORT_DEFAULT isn't selected.
>
> My understanding of the code as it was before I started messing with it,
> is that MMCONF_SUPPORT is intended to indicate whether the facility is
> available at all, and MMCONF_SUPPORT_DEFAULT toggles the actual use of
> it (by Coreboot).  Both src/northbridge/amd/amdfam10/northbridge.c and
> src/northbridge/intel/i945/northbridge.c contains code like this:
>
>  #if CONFIG_MMCONF_SUPPORT_DEFAULT
>          .ops_pci_bus      = &pci_ops_mmconf,
>  #else
>          .ops_pci_bus      = &pci_cf8_conf1,
>  #endif
>
> which, alongside romcc_io.h's
>
>  #if CONFIG_MMCONF_SUPPORT_DEFAULT
>          return pci_mmio_read_config32(dev, where);
>  #else
>          return pci_io_read_config32(dev, where);
>  #endif
>
> is more or less the only use for MMCONF_SUPPORT_DEFAULT as far as I can
> see.  I'll leave it to others to determine if this actually makes sense.
>
> (Having SUPPORT without SUPPORT_DEFAULT gives you an MCFG area that you
> can communicate to the OS (at least potentially), as well as the option
> of explicitly doing individual config accesses via mmio instead of
> ports.  I don't think anyone is doing either of these today.)

Thanks for the explanation.  It looks like the trouble with reserving
a region for MMCONF is related to broken UMA code.

Thanks,
Myles

Patch

diff --git a/src/cpu/amd/car/cache_as_ram.inc b/src/cpu/amd/car/cache_as_ram.inc
index aedb2fd..e21462a 100644
--- a/src/cpu/amd/car/cache_as_ram.inc
+++ b/src/cpu/amd/car/cache_as_ram.inc
@@ -27,6 +27,7 @@ 
 /* for CAR with FAM10 */
 #define CacheSizeAPStack 0x400 /* 1K */
 
+#define MSR_MCFG_BASE  0xC0010058
 #define MSR_FAM10      0xC001102A
 
 #define jmp_if_k8(x)	comisd	%xmm2, %xmm1; jb x
@@ -115,7 +116,7 @@  CAR_FAM10_out:
 	/* Errata 193: Disable clean copybacks to L3 cache to allow cached ROM.
 	 * Re-enable it in after RAM is initialized and before CAR is disabled
 	 */
-	movl	$0xc001102a, %ecx
+	movl	$MSR_FAM10, %ecx
 	rdmsr
 	bts	$15, %eax
 	wrmsr
@@ -136,6 +137,19 @@  CAR_FAM10_out:
 
 	/* Erratum 343 end */
 
+#if defined(CONFIG_MMCONF_SUPPORT)
+	/* Set MMIO Config space BAR */
+	movl	$MSR_MCFG_BASE, %ecx
+	rdmsr
+
+	andl	$(~(0xfff00000 | (0xf << 2))), %eax
+	orl	$((CONFIG_MMCONF_BASE_ADDRESS & 0xfff00000) | (8 << 2) | (1 << 0)), %eax
+	andl	$(~(0x0000ffff)), %edx
+	orl	$(CONFIG_MMCONF_BASE_ADDRESS >> 32), %edx
+
+	wrmsr
+#endif
+
 CAR_FAM10_out_post_errata:
 
 	/* Set MtrrFixDramModEn for clear fixed mtrr */
diff --git a/src/cpu/amd/model_10xxx/init_cpus.c b/src/cpu/amd/model_10xxx/init_cpus.c
index bbaa481..ae1da66 100644
--- a/src/cpu/amd/model_10xxx/init_cpus.c
+++ b/src/cpu/amd/model_10xxx/init_cpus.c
@@ -58,30 +58,6 @@  static void set_EnableCf8ExtCfg(void) { }
 #endif
 
 
-#define _ULLx(x) x ## ULL
-#define _ULL(x) _ULLx(x)
-
-/*[63:0] */
-#define PCI_MMIO_BASE _ULL(CONFIG_MMCONF_BASE_ADDRESS)
-
-static void set_pci_mmio_conf_reg(void)
-{
-#if CONFIG_MMCONF_SUPPORT
-#  if PCI_MMIO_BASE > 0xffffffff
-#    error CONFIG_MMCONF_BASE_ADDRESS must currently fit in 32 bits!
-#  endif
-	msr_t msr;
-	msr = rdmsr(0xc0010058);
-	msr.lo &= ~(0xfff00000 | (0xf << 2));
-	// 256 buses, one segment. Total 256M address space.
-	msr.lo |= (PCI_MMIO_BASE & 0xfff00000) | (8 << 2) | (1 << 0);
-	msr.hi &= ~(0x0000ffff);
-	msr.hi |= (PCI_MMIO_BASE >> (32));
-
-	wrmsr(0xc0010058, msr); // MMIO Config Base Address Reg
-#endif
-}
-
 typedef void (*process_ap_t) (u32 apicid, void *gp);
 
 //core_range = 0 : all cores
@@ -295,9 +271,6 @@  static u32 init_cpus(u32 cpu_init_detectedx)
 	 * already set early mtrr in cache_as_ram.inc
 	 */
 
-	/* enable access pci conf via mmio */
-	set_pci_mmio_conf_reg();
-
 	/* that is from initial apicid, we need nodeid and coreid
 	   later */
 	id = get_node_core_id_x();
diff --git a/src/southbridge/nvidia/mcp55/mcp55_early_setup_car.c b/src/southbridge/nvidia/mcp55/mcp55_early_setup_car.c
index e01630d..6a453cb 100644
--- a/src/southbridge/nvidia/mcp55/mcp55_early_setup_car.c
+++ b/src/southbridge/nvidia/mcp55/mcp55_early_setup_car.c
@@ -267,13 +267,7 @@  static void mcp55_early_setup(unsigned mcp55_num, unsigned *busn, unsigned *devn
 	RES_PCI_IO, PCI_ADDR(0, 6, 0, 0x74), 0xFFFFFFC0, 0x00000000,
 	RES_PCI_IO, PCI_ADDR(0, 6, 0, 0xC0), 0x00000000, 0xCB8410DE,
 	RES_PCI_IO, PCI_ADDR(0, 6, 0, 0xC4), 0xFFFFFFF8, 0x00000007,
-	/* The following operation hangs when performed via MMCFG:
-	     pci_read_config32(romcc): 00010000:0078: 20040000
-	     setup_resource_map_x_offset: 10000, 78: 20040000
-	     pci_write_config32(romcc): 00010000:0078: 19040000
-	     (hang)
-	   Response missing? */
-	/* RES_PCI_IO, PCI_ADDR(0, 1, 0, 0x78), 0xC0FFFFFF, 0x19000000, */
+	RES_PCI_IO, PCI_ADDR(0, 1, 0, 0x78), 0xC0FFFFFF, 0x19000000,
 
 #if MCP55_USE_AZA == 1
 	RES_PCI_IO, PCI_ADDR(0, 6, 1, 0x40), 0x00000000, 0xCB8410DE,