EPICS Home

Experimental Physics and Industrial Control System


 
1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  <2006>  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  2019  2020  2021  2022  2023  2024  Index 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  <2006>  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  2019  2020  2021  2022  2023  2024 
<== Date ==> <== Thread ==>
Subject: Re: Dev lib off-board register access
From: Andrew Johnson <[email protected]>
To: Eric Bjorklund <[email protected]>
Cc: EPICS tech-talk <[email protected]>
Date: Fri, 14 Jul 2006 14:05:14 -0500
This is a somewhat long and rambling message that will only be of interest to those people who work on BSP development and VME driver code....


Eric Bjorklund wrote:

I believe it is hardware caching that we've been talking about. The "software" part refers to system routines in vxWorks (and probably in RTEMS too) that talk to the MMU to either invalid or flush the hardware cache (depending on whether your DMA is reading or writing). These routines are, of necessity, system-dependent, and it would be nice to have devLib implementations so that DMA device drivers could be more portable.

If your CPU hardware properly implements bus snooping then the cacheFlush() and cacheInvalidate() routines are unnecessary, but there are some CPUs that can't snoop memory cycles or where such snooping is broken (the mv172 and mv177 for example) - here the flush and invalidate operations are needed. I agree that it would be a good idea to put an API into devLib to implement these cache operations, and it should also provide equivalents of the vxWorks cacheDmaMalloc() and cacheDmaFree() routines and associated macros CACHE_DMA_FLUSH and CACHE_DME_INVALIDATE. However I think we need to consult with the Linux users of devLib before trying to design anything like this...

There is a related issue which is starting to raise its head as CPU boards get more and more RAM on them - soon we're not going to want to export the whole of the CPU memory to the VMEbus even in A32 space. There are boards available now that have 2GB of RAM, meaning that the RAM extends from 0x00000000 to 0x3fffffff which is a quarter of the 32-bit address space.

I should step back a little at this point and explain that the APS used to use the vxWorks 'sm' shared memory backplane driver to permit communication between two vxWorks CPUs installed in the same VME chassis. As a result of that, our BSPs were configured to ensure that all CPUs exported their local RAM at A32:0x80000000 (when they were set as CPU#0 in their boot parameters), and all CPUs could do VME read/write operations at A32:0x80000000 through 0xbfffffff (the 68K's could access A32 addresses up to 0xdfffffff or higher, but the PowerPCs can't because the PowerPC BAT registers can't map windows as large as the 68K's TT registers could).

Naturally any VME I/O boards we use here that need an A32 address are set up to appear within the A32 range 0x80000000 thru 0xbfffffff. We don't actually need to support the shared memory backplane driver any more, but we'll be sticking with our address mappings for compatibility with those I/O board addresses. The problem is that we're still exporting the whole of the CPU's RAM to A32:0x80000000, and as RAM sizes increase there's less and less of the A32 space left for VME I/O boards. If we were to buy one of those 2GB boards, we'd have no A32 space left at all!

Therefor we will need to limit the amount of on-board RAM which can be accessed from the VMEbus. However there is currently no mechanism for the driver of a board that does DMA to allocate memory that will be guaranteed to be accessible through the CPU's A32 slave window. APS BSPs have included an equivalent API for obtaining memory for A24 DMA operations which devLib's devA24Malloc() and devA24Free() functions can make use of, but it looks like we may need to add an A32 version of this sometime.


Appologies for the brain-dump, I'd be interested in hearing the thoughts of anyone else who has wondered about this issue...

- Andrew
--
Not everything that can be counted counts,
and not everything that counts can be counted.
  -- Albert Einstein
Replies:
Re: Dev lib off-board register access Andrew Johnson
References:
Dev lib off-board register access Rees, NP (Nick)
Re: Dev lib off-board register access Dirk Zimoch
Re: Dev lib off-board register access Eric Bjorklund
Re: Dev lib off-board register access Kate Feng
Re: Dev lib off-board register access Eric Bjorklund
Navigate by Date:
Prev: Re: Dev lib off-board register access Andrew Johnson
Next: Re: Dev lib off-board register access Kate Feng
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  <2006>  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  2019  2020  2021  2022  2023  2024 
Navigate by Thread:
Prev: Re: Dev lib off-board register access Andrew Johnson
Next: Re: Dev lib off-board register access Andrew Johnson
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  <2006>  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  2019  2020  2021  2022  2023  2024