BASIL_NETWORKS


Designed & Made
in America (DMA)

ABOUTABOUTPRODUCTSSERVICESSUPPORTCONTACTARTICLESBLOG
BASIL Networks Blog BN'B | IoT Core Platform Project

10 Mar, 2021

Internet of Things (IoT) Security, Privacy, Safety -Platform Development Project Part-22

Part 22: IoT Core Platform Development;- Peripheral I/O Device Design

The IoT Embedded Core Platform - Microprocessors & MicroControllers Update-
Peripheral Devices Real World Testing -
Continued

"People who are really serious about software should make their own hardware."  Alan  Curtis Kay ( May 17, 1940  )

As we stated in past presentations in this series, development costs are easily exceeded when the performance expectations and in that case performance requirements are not documented properly or, the infamous "TBD".  This forces a direction change "AND" direction changes are commonly not listed as part of  performance expectations.  Sometimes the development "process" is faulty or just plain broken period.  The intent of this series in not to maintain the insanity of over budget development costs but to disrupt it in order to allow the creation of new habits that will give a solid foundation for engineering practices to be successful when starting a development project.

For the new designer that is taking on the learning of CPLD and FPGA design, "Each design completed is experience for the next lever of development."  There are only experiences and techniques that will bring you to the advanced level of applications.

IoT_Index Quick Links

Quick review to set the atmosphere for Part 21:
A full year has past since we posted on this IoT Platform Blog, to be fair, we were waiting for the dust to settle in the CPU arena to see who would win.  Well it is obvious that the ARM processor has won the battle at this time for the next platform.  Of course Intel will always be around and maybe it might even get the number one spot again.  Since the M&A of AMD and ARM the processor industry is undergoing a big change, Apple, Microsoft and others are moving over to the ARM processor architecture for software development.  Oh, by the way! the initial budget was exceeded as we will see in this part, BUT! for the better - of course.  alt 

So as of this writing here is what has been happening in the micro silicon world,   A couple of new embedded processors have emerged from all the Mergers & Acquisitions and many processors have been discontinued.  Performance and features such as higher core speeds and multiple cores make it applicable to many AI (Artificial Intelligent) applications.  Several interface ICs have been released that may help the interface along, however careful consideration has to be taken to insure that the interface will be around for at least five years to accommodate the embedded lifecycle requirements.

For review, the current Core Platform IoT development main focus is two fold, first an educational project development for the entrepreneurial mindset and the project applications of remote sensing and control incorporating safety, security and privacy over a wide range of peripherals including wireless.  This should be kept in mind since we will be designing in some redundancy for reliability and security.

The ITF development Project:
The ITF did create a change in direction, a change in budget and the project time line, however did not change the desired results, this is not only common in development projects but in some cases required to meet the desired expectations.  The delay for this blog was mainly due to the selection of the embedded processor that we will be using for the project.  Over the past two years there has been a lot of supply-chain management issues of embedded processors even though many supported the ARM processor they were all different in their interface capability which caused instability in the industry for selection.

The issues here still, are handling multiple projects, resources available and the changed timeline, just a short change in direction to complete the objective.  A validation that the real time product development process is far from being a linear process from conception to finished product.  That added with supply chain management to avoid redesign shortly after the product is release just creates uncertainty in the development cycle.  

To repeat,  Remember changing direction overnight does not mean changing goals our the final destination, it is just a better way to insure you will reach the desired destination.  With many years of exposure to the design arena as a designer, troubleshooter and mentor the honor of experiencing innovative and passionate creators has lead to the full realization that the creative thought process of an individual is not a linear step function, 1, 2, 3 ...N, probably because electro-chemical based humans are not programmable in the same way silicon based robots are that follow a preprogrammed set of processes, as some may think of engineers  The innovation of the human mind subconsciously is always creating and performing scenarios to find the best solution for the task at hand and "developing habits along the way".

What we want to cover in Part 22:
In Part-16 through 21 we addressed the issue of why we should consider testing of the peripherals (Proof of Design) PoD with a prototype build to insure when we interconnect several of the peripherals the throughput required for the applications will be met.  This allows the opportunity to change the development direction from the peripheral point of view if performance expectations become an issue. Developing a test methodology that will be used for testing peripherals for the platform keeping in mind peripheral throughput limitations. These are new habits being developed at a conscience mind set level that will connect to become the default critical thought process during development.  With that stated we will continue moving forward with detailing the Interface Test Fixture (ITF) CPLD #2.

Updating the IPD (Interactive Project Development) project documentation for the ITF (Interface Test Fixture) to keep track of the development process, "Documentation is a living process during development", a good habit to make!  The reference for the ITF is the functional block diagram Figure 16.1

Lets Get Started:
Some questions answered from our readers
"Again What happen to the series?,  "It has been almost a year from part 21, are you going to continue the series?  
To answer the above two questions - YES the series will continue.   What happened is simple a life TBD slipped in the process and adding a new blog to This IoT Platform Project that we were going to have to cover at some time so we were talked into doing it now, hence: the Wave Phenomenon, Antennas and Radiation blog was created. The blogs plus my daily responsibilities does not allow much if any time to get in trouble, yes, there are some that are happy with this situation.Cool   The main issue as we stated in previous parts was the selection of the embedded processor with respect to all the M&A's of the embedded industry reorganizing for the next generation of processor development, which appears to have shown some smoothing out, a direction with some major players for the next generation.  Here in the lab the past year has been actively creating some hardware and software for other parts of the blogs to give all interested some useful tools for this series that may also be applied to many other development projects as well.

What is next for the Embedded World
What is next in the Embedded Processor world, well, simple, Microchip, AMD, Intel, NXP, TI and a few others will be battling it out for a favorable position in the multi-billion dollar market place.  The merger of AMD and ARM will set the stage for the industry to move forward.  It is expected to see a new set of development tools for the ARM processor as well as a more robust series of cores.  AMD's ARM's M&A did catch a lot of attention for not only the embedded graphics market but also the desktop, Smartphone, and Tablet markets as well.

Keep in mind that all the M&A's are going to stabilize the market for a few years after the release of the new products that will identify the market niche's for each of the competing vendors.  The hobby market will see an influx of the older obsolete / discontinued processors that will be supported by some older software as the new cores are released.  Our IoT platform will be using the latest releases of cores to give at least a five year cycle if not more.   We will release the selection of the processors and why we chose them later in this part.

Changes In Supply Chain And M&A's:
OK, a lot has taken place in the past year with components and M&A's that have effected all manufacturers of the companies that used these components.  This is how it effected BASIL Networks Research, the IoT Core Platform blog development and with a little bit of a critical thought process other manufacturers of products that use the effective components.

To start a quick brief of BASIL Networks, 25 year experience dealing with Intel Corp Development group.  To cut the chase, Intel is well known for turning silicon every eight to 12 months and put the old silicon in "embedded or other market place department"   We presented this a few parts back about what happens with many M&A's and the products being discontinued.  

Discontinued Parts From the M&A
All this started with a PCN (Product Change Notice) in December 2020 from Intel Corp. discontinuing the entire MAX II line of CPLD's as well as over 500 other devices from several of the FPGA product lines.  OK, This is Intel's absolute right as a manufacturer to discontinue any product they want, when ever they want and when they want period.  The right of free enterprise, however there are always consequences to every action.  This is how it effected BASIL Network, PLLC and this blogs IoT Core Platform development series.  This is a great opportunity to document cause and effect of M&A's industrial changes.

Embedded Process industry expectations for over the past 20 years, Altera and others have kept the product line intact for the past 15 years well over the industrial standard of 5 years. This was great for small niche companies that were able to compete successfully.  The other companies that followed this path were Motorola, TI, AMD and many more.  Intel always paved its own path, however since the new major players are competing for a top level in the $40-$50 billion market of embedded changing and discontinuing silicon at a whim is no longer going to build the trust required to obtain this share.

The design used for the following test is the same design presented in this blog for high speed digital I/O direct to high speed memory which was designed back in 2001, yes a few years ago.

The Real Effects On Manufacturers
OK, some reality dealing with these changes!  For end user products that use MAX-II devices which there are many, however not as many as CPU's numbers, adding to that the fall in small businesses over the past two years due to lock downs etc. in those aspects the demand is low.  So no big deal just switch over to the MAX-V line which is still active and is a bit less expensive.  Well, it is never that easy to just switch components out these days and in this case this is what product designers and manufacturers have to consider.

As for this blog and the ITF, BASIL Networks, has enough part to build a few of the ITF for testing and use, however we will discuss this shortly.

Software Before vs Software Updated After M&A
Software is always a big issue when it is developed by different manufacturers to be used on the same product series, back to the opening quote.  I am not sure where the updated software for these CPLD/FPGA devices was developed and it did not really matter since it was from Intel Corp, until these tests were run.  For those of us that are able to program in VHDL if it compiles and runs with 30% less logic remaining then is should be consistent or close when moving from device to device in the same family; VHDL code is used in this design.  The majority of the complex blocks were compiled with the same VHDL code used with previous releases and followed the expected consistency of percentage used per family of chips.

OK, we discussed the new Intel Quartus Pro Lite CPLD, FPGA development software used a while back and it is time to update the software after a year or so.  Since this was a reuse design and developed with Quartus 9.1 which is way past its support life and also does not support MAX-V the we are now forced to use since the MAX-II is discontinued, the last release of Altera Quartus, 15.1 prior to the M&A was downloaded.  All still worked with both MAX-II and MAX-V as we will see below in the comparison table taken from the compilation report directly.  This design was saved from release 9.1 then copied to a separate directory, opened in release 15.1 and converted and compiled on the MAX-II with no changes, however on the MAX-V, the line "set_global_assignment -name POWER_EXT_SUPPLY_VOLTAGE_TO_REGULATOR 3.3V" from the ADC_CHAN.qsf file had to be removed or changed to 1.8V for the MAX-V in order to allow for the 1.8 volt core voltage, we just removed the line as to be logic compatible.

As shown below in Table 22.0 below the compilation of both series MAX-II and MAX-V are very compatible and as expected the MAX-V is a bit faster as well for the similar devices.  Both family lines fit the design easily and is a little over 60% utilization but still well within the percentage for minor changes or additions for a new layout.  OK, since changes are going to be made, our engineering brain opens up the big gadget box and out pops the Engineering-Wish-List that was put aside during the initial development years back. Cool

MAX II   EPM1270T144C3  Quartus Rel  15.1

MAX V  5M1270ZT144C4  Quartus  Rel 15.1

  • Flow Status Successful - Sun Mar 07 07:45:43 2021
  • Quartus Prime Version  15.1.0 Build 185 10/21/2015 SJ Lite Edition
  • Revision Name   ADC_CHAN
  • Top-level Entity Name   ADC_CHAN
  • Family  MAX II
  • Device  EPM1270T144C3
  • Timing Models   Final
  • Total logic elements  842 / 1,270 ( 66 % )
  • Total pins  111 / 116 ( 96 % )
  • Total virtual pins  0
  • UFM blocks  1 / 1 ( 100 % )
  • Flow Status Successful - Sun Mar 07 07:42:34 2021
  • Quartus Prime Version  15.1.0 Build 185 10/21/2015 SJ Lite Edition
  • Revision Name  ADC_CHAN
  • Top-level Entity Name  ADC_CHAN
  • Family  MAX V
  • Device  5M1270ZT144C4
  • Timing Models  Final
  • Total logic elements  842 / 1,270 ( 66 % )
  • Total pins  111 / 114 ( 97 % )
  • Total virtual pins  0
  • UFM blocks  1 / 1 ( 100 % )

Table 22.0   Compilation of MAX-II vs MAX-V using  Quartus 15.1

We kept the same design that was compiled on release 15.1 created another new directory and copied the complete folder used in 15.1 to the new folder, opened the project with release 20.1 and did the conversion and ran the compiler.  Table 22.1 below shows the comparison from Release 15.1 and 20.1 compiled comparisons.  The red highlighted were the errors from the new 20.1 that stated the design would not fit in the selected device.  A group of 8 pins were added by the compiler for some reason as well as the extra 31%  of logic used for the same design.

MAX II  EPM1270T144C3  Quartus  Release  15.1

MAX II  EPM1270T144C3   Quartus  Release  20.1

  • Flow Status Successful - Sun Mar 07 07:45:43 2021
  • Quartus Prime Version   15.1.0 Build 185 10/21/2015 SJ Lite Edition
  • Revision Name   ADC_CHAN
  • Top-level Entity Name   ADC_CHAN
  • Family  MAX II
  • Device  EPM1270T144C3
  • Timing Models   Final
  • Total logic elements  842 / 1,270 ( 66 % )
  • Total pins  111 / 116 ( 96 % )
  • Total virtual pins  0
  • UFM blocks  1 / 1 ( 100 % )
  • Flow Status Flow Failed - Sun Mar 07 07:11:54 2021
  • Quartus Prime Version   20.1.1 Build 720 11/11/2020 SJ Lite Edition
  • Revision Name   ADC_CHAN
  • Top-level Entity Name   ADC_CHAN
  • Family  MAX II
  • Device  EPM1270T144C3
  • Timing Models   Final
  • Total logic elements  1,227 / 1,270 ( 97 % )
  • Total pins  119 / 116 ( 103 % )
  • Total virtual pins  0
  • UFM blocks  1 / 1 ( 100 % )

Table 22.1   Compilation of MAX-II=Quartus 15.1 vs MAX-II-Quartus 20.1

Running the same test on a MAX-II device selection the differences were not expected to be this far off.  Keep in mind that the memory requirements are 2GB RAM for the MAX series and un to 64GB for selected FPGA's required for release 20.1, our system is a quad 3.3GHz Intel processor with 16GB RAM Win10x64 Enterprise.  The test uses the same design file and ADC_CHAN.qsf setup file for all the tests, all the pin assignments were reset to allow the compiler to select the pins.

Here in the lab we have been running releases up to and including 15.1 on 8GB Windows 10x64 and 4GB on Windows XP for the Cyclone FPGA and MAX-II series devices for many years.  The lab systems now run 16GB on the Win 10x64 systems and the majority of software has been upgraded to run 64 bit.

BASIL Networks Review of the Development Supply Chain
OK, this situation now forces us to look at the future development for our research projects and the instrumentation that will be required to prove out findings.  This being said there are only a few companies that manufacture FPGA's and CPLD and we will look at the remaining companies and review their product lines.   This means downloading the latest software and purchasing a few devices to test for functionality for our research.    This definitely changes our direction as well as add a lot of time to the project.

Open Source vs Licensed vs Proprietary Software:
OK, why are we jumping into software now?  Well, it is always a good idea to look at what we want to do with the hardware, keeping in mind that some sort of software has to communicate with it, hence:  the 16 bit digital input and how the data is to be stored.  Since we are looking at a typical 10ns digital data collection on a BUS structure it would be a good idea to discuss some software to display this data logically.  OK, the old saying "when you have two or more lines of code there exists a troubleshooting requirement."  BASIL Networks over the years, accepted separate software contracts for other designers hardware only to come to a very fast realization that our quote of the day holds very true.  "People who are really serious about software should make their own hardware."  Alan  Curtis Kay ( May 17, 1940 )    So that is what we do, a full turnkey development, hardware software, documentation, training.  The decision was made very shortly after the company was started and our first big loss of what was supposed to be a simple software contract ended up being a redesign and coding that made it easy to update the company policy on only software contracts.  Hardware contracts work the same way when the outsource software company wants the hardware to be changed.

Software & Data Verification / Validation
One of the big issues today is with "software" used to setup and analyze data being collected,  Rewind! I meant over the years including today.  In the beginning even before the IBM-PC release in August 1981 there have been many software packages released to analyze research and test data collected.   This was accomplished by renting computer time all the way to purchasing "mini-computers" like DEC, Data General etc. to perform analysis to verify designs and research findings.  At that time it was very expensive to purchase complete hardware and software systems along with the analytical software compilers required to create analytical programs. BASIL Networks still has the licensed copies of Fortran, Pascal, C, BASIC that runs on MSDOS on the IBM-PC and compatibles, yes, 16 bit versions.   As the desktop PC evolved packages like MathCAD, MatLab, Maple and several others entered the PC Desktop market, however they were still expensive ranged from $1000 to as high as $5000 and were still 16 bit versions.  Here in the lab we purchased a couple of them only to realize the limitation along with the fact that supplemental software had to be written to meet the labs development needs.

The next issue we ran into in the lab was all the third party data collection hardware on the market have their own data file formats and to be fair they do offer some setup and display software that meets about 75% of the markets needs.  Our research lab is generally not part of that 75% and when you are researching new areas having limitation does not help to get accurate results required for proof of concept and design used for obtaining patents.  When presenting research data results, it is extremely important to be able to present the raw data, the equipment used along with all the parameters and setup on how the raw data was collected.  Presenting the final results through analytical algorithms is a great tool to allow the researcher to present how the data was manipulated, however this would all fall short if the raw data is not available to prove the findings.

What we realized over the years in the lab that it is not that difficult to develop analytical software with the easily available off-the-shelf math packages in C, Fortran and other languages, for the record, python language is not a math package software and was not designed to perform math gymnastics. It is however a very good process control package which may be linked to any external hardware for math gymnastics.  What makes this more attractive is that we are able to easily link it to our standard interface format for the display software already developed.  Yes, it does require you to learn software programming which is not very difficult.  Over the years we have used Assembly, BASIC, Fortran, Pascal and C with all the variations.  Assembly language is still used in the lab for driver development and interfacing to custom hardware as well as using some special interface IC's for specific bus types when needed.

16 Bit Logic Analyzer Display Controller Software:
So, here we are, what's next?  For our readers that have been following us over the past few years, we have decided to add some of our lab development software that has been and still is priceless during our development contracts.  Our main concern was how do we offer free software that is useful to the end user without adding the old paranoid feeling of  "if it is free you are the product" and what hidden stuff is in the application that sends info back to the manufacturer?  OK, lets address this issue now and put it to a rest.  BASIL Networks, PLLC software for this series follows under a separate BASIL Networks Blog Series End Users License Agreement (EULA).  This EULA will also be embedded in the Software IMAGE_Bitmap_16x16_ABOUT.jpg "About' page the application.  Oh, by the way - There are No PopUp Adds or any other annoying PopUp stuff,  Below is the Software EULA in a nutshell unless otherwise stated.  Our intent is to release an application that is useful to use for a long time, enjoy, grow and contribute.  It is always good to know the EULA before you open the box!

OK, So what was done to this software to extract it from the IBPD system Integrated Development Environment (IDE) ?  The modifications are listed below, in a nutshell which include all the links inside the IDE in order to make the application stand alone.  The entire system features are explained on the website that will save duplicate typing. The internal links common to all the other application packages within the IDE, https://www.basilnetworks.com/Products/Programmers/IBPD-SYS/CLADS/IBPD-CLADS_StartupMenu.html,  https://www.basilnetworks.com/Products/Programmers/IBPD-SYS/LAD/IBPD_LAD_Dialogs.html.

OK, there is another limitation, or feature that has been removed for the free version and that is the encryption feature.  The encryption feature is part of the full IBPD systems Integrated Development Environment (IDE) that was developed over the years.  The encryption is a two fold methodology incorporating a stand alone standard 256 AES encryption (not part of the Win 10 AES internals)  as well as a proprietary Complex Obfuscating Methodology Block (COMB) to be discussed in a later series on encryption and security.  A quick list of the removed IDE features is as follows:

The hardware drivers and links were removed in order to make this a universal LAD controller that may be interfaced with existing digital data collection devices both old and new.  The external data interface is discussed following the main software in this part.  The LAD controller is a stand alone application that does not require any installation, just copy the files and directories in a top level directory and run the BPLAD.EXE file directly from the top level directory or create a desktop link to the top level directory that has the application.  The sub-directories are required to run this software and are referenced in the application to startup, save and organize the application.  You may also run this directly from a USB drive also as long as the subdirectories are on the USB drive.  One of the main features is the user has control of where the directories will be placed for each project and may load the directories for the current project at startup.  Not all the project features are incorporated into the stand alone application however, the directories may be manually setup for each project then loaded as required for the current project being worked on.

This is an extracted application from our internal IBPD (Interactive BUS Protocol Development) System that we use here in the lab that has been upgraded as required over the past 20 years to meet development requirements.  Yes, the original development started in 2000 as small separate applications where we soon realized that keeping track of software and projects with many applications has created confusion and time lost in the development process.  Hence: the IBPD (Interactive BUS Protocol Development) System was created and has evolved into a fully flexible innovative IDE.  This blog has opened the door for an opportunity to upgrade and pull out our "Engineers Wish List" for the next generation of challenges in the lab.  All the software presented here will run on Windows XP sp3 through and including Windows 10 x32/x64 without any special installation process, just copy to a top level directory and run, to remove it just delete the directory.  We will cover interfacing and other features after we present the dialogs and the ease of use.

To get a registered copy of this LAD controller software just click on the button below and fill out the registration form, a Link will be sent to validate the e-mail address.  You will receive a link to download the software registered to the information stated on the registration form.

LAD_Software

The Logic Analyzer Display (LAD) controller is a full 16 bit display controller whose file structure contains all the necessary parameter information that is attached to the data file either RAW or converted from an external users data file to necessary levels being sampled.   The main purpose of the application is to allow the user to organize digital data collected with identification parameters that may be used to validate, troubleshoot and patent hardware that the data was collected on.  Figure 22.0 below shows the MAIN MENU for the LAD controller.

The Main Menu below shows 8 button group that the user may easily add links to any program that are simple convenient shortcuts that may link other programs that are used for the current project being worked on.  The buttons are configured through the Configure User Buttons on the main menu.  Command line links and commands are also allowed with these buttons and easily programmed.   Multiple configurations may be programmed and saved with the users project directory for each project.

A simple RTF editor used to read/write/edit RTF files that are linked to the data file to characterize and describe the data in the file for clarity and validation.  The data file RTF header allows a total of 65,000 bytes to be added to the data file when the data file is saved.   The project control has been disabled in the free release, however the user may create project directories and load them through the eight user buttons that will load the startup parameters from the saved project directory.

The explanation of the features for the LAD Database below https://www.basilnetworks.com/Products/Programmers/IBPD-SYS/LAD/IBPD_LAD_Database.html.  Trace colors and the Digital 16 bit word recognition pattern is maintained as well as the ability to have up to 256 different configurations of the data collected or imported to a standard file format.

LAD_MAIN_MENUFigure 22.0   ITF 16 Bit Logic Analyzer Display Controller   MAIN MENU

The purpose of the LAD database configuration is to keep all the parameters associated with the data collected  in one convenient place for each project.  All parameters are setup prior and are ready to be attached to the imported data file of the LAD Controller and saved to the SS_LAD sub directory setup from the users top level directory.  The user also has a 65000 character comment buffer that is through the RTF editor that is also attached to the data file and may be used for explaining any special features or configurations used during the collection of data.  A PDF file is also part of the project directories that is used to store any custom test fixture setup diagrams, processes and manuals used to collect data including any special components used.  This allows users to review data in years to come an get accurate information for reuse or verification processes.  Figure 22.2 below also shows the user may label each bit trace with a specific name, color, groups, assigned timing ID bars, Number of Samples, ands a 16 bit trigger pattern to identify the data being imported.   The Database files created from this dialog is saved in the default directory LAD_DBX shown in the directory structure in Figure 22.1 below.

The directory structure is a simple top level and sub-directory as shown in Figure 22.1 below.  The directory structure and the program startup files are small enough that duplication this structure for each project is simple and organized.  We have used this for many presentations and it is simple and effective and runs on a thumb drive.   There are three sub-directories off the Top Level BPLAD directory that are the Startup Directory SYS_LAD, the main application Help Directory LAD_HELP, and the project directories for the current project LAD_xxx.  Since the project and encryption features have been removed making this a stand alone application an duplicate for each project would be required.

LAD_Directory_Layout
Figure 22.1   ITF 16 Bit Logic Analyzer Display Controller  Directory Structure

The Configuration database allows the user to assign up to 256 unique configurations per Data File List block of data files an select any one for data importing.  The Data File List link that shows the number of different data imported files for the displayed configurations allow the user to maintain a common configuration setup for active projects and data collection.  The data file will incorporate the selected configuration to the imported data file and display the configuration when selected as shown in Figure 22.4 allowing easy organization and display of parameters used for the data collected displayed in the LAD graphic display of figure 22.3.

The user may have as many configuration databases required per project and these may also be sorted separately and placed in a project directory's data files for convenient loading / saving as desired.  The trigger pattern is a 16 bit pattern that consists of any combination of clock or level states will be displayed on the LAD graphic display shown in Figure 22..3

LAD_Database
Figure 22.2   ITF 16 Bit Logic Analyzer Data File Header Software Setup

The explanation of the features for the LAD Display below https://www.basilnetworks.com/Products/Programmers/IBPD-SYS/LAD/IBPD_LAD_Traces.html.  The update for this display now includes the ability to display trace data up to 16 Meg samples and sample rated as fast as 10 ns.  The graphic display shows the state of each bit and the hex 16 bit word.  There are many inexpensive logic analyzers on the market today to data at various sample rates and use the USB port to scan the data to the file system.  Some of the software is quite clever addressing the serial protocols directly at their adopted speeds.,  The ITF however is a straight forward parallel bit sample rate capture device that will sample data at 5ns for the full amount of SRAM installed and then display it at the actual fixed sample rate collected for the full memory.  Keep in mind that SRAM of the 10ns and faster is costly in some cases more than the device stating it can collect at 10ns for 256Meg words, just saying!

LAD_Graphic_Display
Figure 22.3   ITF 16 Bit Logic Analyzer Graphics Display

The complete file structure for this display is given in FileStructure_LAD.h  and is the same for the 16 channel Analog display controller as well which we will discuss in another part of this series.  The import file structure is also included in the header file in order to import any file data collected.  A free copy is available for the asking, all that is required is a valid e-mail address and registration,  a link will be e-mailed do download the registered software with all instructions.  Once the file is saved it will automatically setup the parameters that was saved with the file and automatically displayed when selected from the Data File List Dialog shown in Figure 22.4 below.

The data file organization for this applications allows the user to organize all the data files in a test sequence for the current project in a single Data File List for selective displaying graphically.  The files default to the BPLAD\SS_LAD\LAD_LAD directory,  however the user is free to save these in any file accessible to the system.  This allows the user to organize data files and document files per project in an organized directory structure for easy access.  The emphasis here is on the header files of each data file containing all the required information to identify the data collection, test setup and any special handling of the instrumentation used for the data being presented.

Data_FIle_List
Figure 22.4   ITF 16 Bit Logic Analyzer Data File List Organization Display

ITF Peripheral & Software:
What a lesson learned about reuse that fits the presentation we gave a few parts back, this task was a challenge and for the better as well, since we did save some time in rewriting and creating all the dialogs.  In the process of removing the LAD application from the IBPD system and all the older drivers as well our initial presumed budget and time went completely in the deep six.  A lesson in reuse from the past, we did use about 35% of the code which included the dialogs, about what was expected from our past presentation on reuse.  To start the removal took much more time than just cutting and pasting since all the security features were tightly integrated and checked during several operations to prevent hacking.  Then there were the typical engineering wish list "like to add this" process of which actually happened, which then lead to a change in the ITF hardware design of course.  We will cover the hardware changes following this section.  Since this is a software application that requires the user to interface some external data file collected from third party hardware to import a data file an import function was also added for users to just click and import data from other data collection hardware.  So in the process of adding the import function the main body was updated also which when all put together added a lot more time and cost to the development process.  Again the TBD development surprises!

This application does not have any device drivers and will accept standard stream type data files as shown in figure 22.5 below.  For a simple update about data files, here in the lab we use the standard C, C++ or C# language file open [ FILE  *fopen(const char * filename, const char * mode ) ] commands to create a data file whenever possible, with embedded systems collecting data in a SD of FLASH memory we generally use Assembly for full control of the embedded system.  This stream type of  format just packs the data words one after another with no separation characters like CSV or any other delimiter.  It is very simple to open this file with a hex editor to get the offset of the first data word and put it in the HEX offset field of the LAD Import dialog to import the file in a single click.  Figure 22.5 below shows that the file offset is HEX 3C where the first 16 bit data is located.  The number of 16 bit data samples will be calculated at the end of the data import process and placed in the LAD Controller header file if the standard header format is not used and only the offset feature is used.  Logic Analyzer data files are usually a stream of data controlled by a fixed clock sample rate in order to monitor noise spikes or unwanted random data patterns so we only put the standard offset to remove the header of a data file, of course the parameters are setup in the database configuration prior to be able to keep a consolidated record of the collection.  As we move into the analog display side we will get to the other type of data file structures that have delimiters and multiple data values per row / column as the series moves forward.

IMPORT_DISPLAY
Figure 22.5   ITF 16 Bit Logic Analyzer Data File Import Display

The other method of importing a data file is to create an intermediate file from your custom users data file which would require a small amount of programming.  There are so many different file formats for saving the data that it would be easier to just give a program that allows the user to just configure a data transfer program that addresses their file format.  The end result that is required to use the LAD Import File applications is given below.  With that in mind we have included the file structures for those that want to create their own transfer program to the preferred Import file format below.

typedef struct  _IMPORTDATAFILE
   {
        int     DataSize;                  //- Data Number of Words (32bit value)
        U16 LaData[0x1000000];   //- Data Array [1-16Meg words] (16bit value)
  }IMPORTDATAFILE;              //- ImportLadFile   //- LADatFile structure ID

The import function for this release is very basic and may be used to bypass the header information and only transfer the data to also keep track of the original data file.   Figure 22.6 below is a typical display of a Hex Editor showing the first 16 bit data in this file starting at offset Hex 000003C which is the same file offset used in the dialog in Figure 22.5 above.

HEX_FILE
Figure 22.6   Hex Editor Shows A Raw Import Data Files HEX data

OK, there are many hex editors that are free and very useful, in this example we used UltraEdit from IDM a commercial release since we have been using this application for the past 20 years and it suites our need very well.  The free hex editor that we actually tested and sometimes use in the lab on test systems is NotePad++  with the Hex editor plugin for those that do not want to purchase a commercial version.  Keep in mind that BASIL Networks policy is not to endorse any products on this blog nor do we accept any monetary collaboration for products mentioned in this blog, this allows us to be completely unbiased and without conflict of interests, strictly scientific.

The software as we presented above is a general purpose display controller for a 16 bit data file of 2N points to be displayed as a Logic Analyzer Display (LAD) controller one bit at a time.  This LAD controller will handle serial protocols easily up to 10 ns clock rates for this free release and as fast as 1 ns for the commercial release device.  However, for this free application the LAD will handle protocols like  I2C, SPI, Microwire, Onewire,  Quad SPI and any 16 bit data collected easily.   

OK, what are the limitations?  The only limitation which is really not a serious limitation for this free application is the maximum data array size set at 16 Meg x16bit words which should be enough to handle any 16 bit Protocol in the embedded world.  If you were to price logic analyzers that handle 16 meg words x 16 bits or more at 10ns clock rates you will find the price tag anywhere from $50.00 to over $1000 depending on the software and hardware, keep in mind that stable periodic sampling at very fast rates are more difficult to maintain with DRAM that require refresh cycles than with true SRAM that do not required refresh.  Most of the inexpensive logic analyzers incorporate FPGA's and DRAM and lower the transfer rate as the number of channels increase also.  

Adding Features After The Fact Development:
The TBD syndrome effects:
OK, TBD's were addressed several times throughout this series so why now?  Well during the past year in the lab while using the IBPD software the old "Engineering What If " enters the picture since all the upgrades with the software and Engineers of course want a silver bullet that takes a very long time and lots of budget resources.  So, modifications of both the CPLD design as well as the software as mentioned above are on the table.  Needless to say this took some time and delayed the project again since a discussion on just what to add to update the package, so once again out came the engineering wish list, (TBD's at its best).  Once again it is easy to leave out requirements / want to have features even when you know what you are working on.  "Research is what I'm doing when I don't know what I'm doing"  - Wernher von Braun

The following features were added to the design of the digital interface and the analog interface.  These simple added features, of which the CPLD's are able to handle with ease which did cause a delay in the development as well as a redesign of the ITF, for the better though.Cool  To take something away from this blog this does present a fair experience on the "TBD syndrome."  In Data Acquisition some of these changes are not considered unreasonable additions however,  gaining experience on TBD popup delays during project development - priceless.

The addition of four A/D Channels:

The Digital I/O channel -

A brief comparison of the CPLD#1 A/D design and CPLD #2 High Speed Digital I/O Design.  Both incorporate a SRAM buffer to collect data at a periodic programmable rate or manually triggered externally.  Our engineering wish list discussed the possibility of a multi-core embedded ARM processor and a full DMA controller to be able to collect much larger block of data through a DMA (Direct Memory Access) method, however this was not an absolute requirement the pro's and con's did not balance out to incorporate this since the processors requirements for controlling other parts of the processes would delay the DMA cycle since RISC processors are not allowed to perform DMA in the middle of a majority of their instruction sets that involve memory transfers.  We will touch on this later.

However the design parameters are completely different, for one, the high speed digital I/O which requires a 10 ns static RAM or faster and the A/D Converter is much slower 1.0 us conversion time so a typical SRAM of 55ns is sufficient and much less costly.  A discussion of adding a double/triple/quad buffer SRAM to obtain a 1.0 ns (1 GHz) sample rate was discussed however it was decided not to add that type of high level change at this time as well it is not required for our current research projects in the lab.

The PCB design went from a single PCB to multiple PCB's which is OK, however the budget for PCB's just tripled to say the least.  The daughter boards for this design allows us to add CPLD's, embedded controllers and other logic as needed for various applications and for this design the daughter boards contains a duplicate CPLD and a higher speed memory (8ns) SRAM x18 bit where the 2 remaining bits are control bits for additional features we will be adding in the future.  High speed SRAM in the 8ns arena tend to be more costly since the demand is lower and applications are less, however for the Logic Analyzer they fit the application.

ITF PCB Layout:
OK, now for the ITF PCB shown in Figure 22.7 is the first prototype or Proof of Design (PoD) and may be used as a stand alone A/D with the interface to add a Logic Analyzer daughter board and an extended memory daughter board.  The 144 pin QFP CPLD and the TSOP memory are common since the memory is available in the same package to maintain chip replacement in-house.  Since this is not a ITAR design and is for educational purposes shopping around for the best PCB fab house is suggested and there are a lot of them globally.

The PCB below in Figure 22.7 is the actual ITF design, a 4 layer PCB with two trace layers and two power layers, 0.008" trace and space with the smallest plated through hole 0.015" to make manufacturing yield higher.  We also added a few more channels and made it a four channel 16 bit digital with a four channel analog with programmable gain for testing other analog and digital devices.  The design incorporates a USB 3.0, 2.0 interface on the first board to make it a full stand alone board, however the next revision will be not have this and will be using a separate embedded processor at the top level to address all types of network interfaces and daughter boards independently.  The TSOP-I SRAM will interchange high speed SRAM up to 4 Meg x 16 bits per chip per channel for the analog inputs.  The 144 pin CPLD is a MAX-II  EPM1270T144C5 is available from several suppliers and is in full production.  The four 26 pin headers are for a daughter board that hold additional circuitry for the digital I/O and connectors as well as for future use as required.  At this time it is just a 16 bit digital I/O and the daughter board will incorporate the digital buffer IC's.  We will be assembling this board in the lab since it has no BGA components and can easily be hand soldered, one of the main requirements for smaller companies in-house test equipment.

PCB_4Layer
Figure 22.7   ITF PCB  4 Channel 16 Bit Digital And Analog Input

A/D Channel vs Data I/O Throughput 
A quick comparison of the two technologies, the A/D Channel is a 1.0 us Sample rate via a 50MHz 16 bit  (20ns x16) SPI type serial interface.  The actual A/D conversion time is approximately 700ns which allows ample time for the serial data to be transferred and put in the SRAM on board without missing any bits or points until the number of samples has been obtained.  Another A/D we looked at is the MAX11198ATE+T is a dual 2Msps A/D one clock with dual data serial out, the thought was a simple separate Analog trigger independent of the data collection and may be used for monitoring as well, this is on the Engineering Wish List at this time.  This design however, does not have the dual 2Msaps dual A/D converter design it incorporates the AD7980 Single channel A/D chip.

The Parallel 16 bit interface in CPLD #2 used for the Logic Analyzer is different since we require the full (10ns) 100 MHz x16 bit transfer rate without missing bits.  So if you want a true 10ns sample rate you have to lock the data sample within 5ns which is OK for the CPLD we are using, however the memory access should be less than 10ns to be safe, say 5ns to 8ns SRAM would be better.  This simply states that the full time cycle to write to memory is less than 10ns in order to meet the 100 MHz transfer rate.  The faster the SRAM the higher the cost also there are a smaller selection to choose from when designing with 10ns or faster SRAM devices.  There is a list below of the selected high speed SRAM devices that are available for this project at this time.  Also you can search on Octopart and others like SnapEDA and several distributors for these parts is a good place to start.  For a quick reference a 450ps (0.45 ns) SRAM 4Megx16 is only around $130 plus per IC in a 165-ball FBGA.

Peripheral vs. Functions
When designing any product including test fixtures the Peripheral vs Function discussion will always arise to insure the best price per performance is addressed and not impossible to achieve.  With the technology advancing at such a rapid rate ICs are being release on a regular basis to eliminate support ICs as well as reduce design and testing time.  There are pros and cons with all designs and they have to be addressed.  The approach we are applying in the development of the ITF is flexibility, programmability and separating the front end sensing to separate IC(s).  Of course since this is a test fixture and we are modifying the original design the "Engineering Wish List", will always surface -hint- we request the input of a business managers sense of influence to insure we do not fall over the cliff in time and budget.Cool

How to Obtain the LAD Software and a Finished ITF:
To get a registered copy of this software click on the button below, Fill in the form, a Link will be sent to validate the e-mail address along with a link to download the software.  

LAD_Software

Our plans when the ITF is finished, is to offer an ITF to the public with many more features than the one being developed for this presentation, however that is still on the table.  We already completed two different PoD products to get ready for manufacturing and offering custom development for contract manufacturing companies / entrepreneur small company that want to setup a development test base for future and present development contracts.  Please use the BASIL Networks Contact Form to be put on a mailing list when we are ready to supply the manufacturing prints if you are interested in purchasing the entire system manufactured and tested.  The instrumentation developed in our lab is specific to the needs of the current research projects being studied and not necessarily a full drawn out market research for competing in the market, however there are times when the instrumentation really fits a market niche.

Sections

SUMMARY:
So, this has been an interesting part for the IoT Core platform Development.  The experience of having the selected components discontinued in the middle of a design is real and over the 40 years that BASIL Networks was active this only happened 4 times all with Intel rolling over silicon and letting everyone know that last minute with a short time to do a lifetime buy which we will not do here.  We are actively looking at other companies to form a collaboration with to help prevent this from happening again.   CPLD's and FPGA's are here to stay as well as the embedded processors so we should look at other ways to collaborate to compete in the industry.

We see how easy the "TBD" acronym gets out of control and just seems to tangent into its own entity of development and seems to manipulate all aspects of the design.  BASIL Networks has been in business over 40 years and we have very slowly learned the hard way not to just put TBD's into a contract without specific expectations and discussion on how these would effect the overall performance.  If the performance interaction is un clear then the contract must be kept open for discussion for that area an performance and price must be allow to be negotiated to cover the expense.   Yes, This sounds like a bad leverage for competition, however, it can be much more costly when you are paying for a development when you should be getting paid for it.   Yes, there are competitive companies that will make statements that they will do that for a fixed price even though you know that it cannot be done for that price!  It is up to you to not only maintain your integrity but also to instruct the customer to bring their knowledge level to a higher understanding.   Remember it is ok to break even once in a while but to maintain a business and protect employees you must maintain a profit.

Since it has been a while since our last presentation, a catchup and a short sidestep from the design would be good to get back to the swing of things;  this series will continue with the IoT Platform design and the ITF design.  After presenting the Logic Analyzer Display controller we thought it would be a good learning tool to give out the software that will be used with the ITF.  The LAD controller is flexible to display any digital bit display that is useful in checking a number of serial data protocols or word data collected in a file.

Updating and Modifying CPLDs / FPGAs
Here in the lab we have been very fortunate until just recently that our internally designed and developed test fixtures have stood the test of time for 15 plus year.  We have now updating all our test fixtures, basically redesigning them due to discontinued components and data collection software to run on Windows 10x64 Enterprise.  We still program using win32 for some of the more common applications and in the process of testing out programming with win64 which is an interesting and challenging endeavor to say the least.  For our older ISA Bus peripherals, obtaining a motherboard with PCI and ISA slots that are compatible with older boards and run Windows 10 Enterprise x64 presents a small challenge but are still available.  The best approach is still to design the test fixtures to meet our needs with components that are anticipated to be around several years.  Since Intel has acquired Altera several devices have been discontinued, however the ones we have picked have been discontinued a few months ago.  Also the software has several changes due to licensing transfers during the acquisition that we are still dealing with.  This discontinued product list has caused a setback in timing as well as a budget over run and will take some time to recover.  These are the Oops of a TBD and of course the M&A syndrome.

CPLD Register Identification
Since we neglected this because of the Re-use factor the thought was since all the matrix was already defined prior all that would have to be done is reassigning the bits for each register.  Well that did not turn out as expected and it very seldom does, that lead to the re-mapping of the register set as well.  The redesign became obvious when we started modifying the design realizing that it was over 50% change.  At that point purely by experience it was decided to just start from scratch and layout the CPLD to fit the application with a few spare I/O pins for future growth.  In this reuse case it was fortunate that the redesign was recognized early before many man hours of troubleshooting and development were lost.   

If we are to use the latest release 20.1 of the software we will have to seriously look at the design and determine if we should use the new MAX-10 FPGA along with the cost and time impact.  CPLD's and FPGA's are very competitive in the marketplace today and if you decide to switch manufacturers the learning curve may be steeper that expected.  Until the root cause of the increase in pin assignments and the increase in logic elements used in the latest release is understood, we will use release 15.1  for the Proof of Design (PoD) at this time.

Final Note
Since repetitiveness is the mother of retention, "Changing poor engineering habits are difficult however not impossible to correct".   Humans are very flexible they all have the ability of learning anything with applied effort, the only impasse is the mind set that if negative will defeat any attempt to grow and instill fear of learning.  The key is to acknowledge the initial behavior, no, it will not change overnight - it took a while to become rooted.  "TBD is a typical behavior when the desire to obtain a contract exceeds the common sense."

Bringing the development behavior to the surface and acknowledging the behavior is the first step in this series to bring the development process to a winning level.   The expectations of this and other series on the blog is to encourage the self behavioral changes as well as present a project that is useful for many applications.  Behavioral modifications is a personal and private process that takes time which requires trust within ones self.

As the series progresses the author, Sal Tuzzo will be available for discussion through the BASIL Networks Contact Form for those that want to apply this series to conduct their own experiments.  I will always be appreciative for the private comments sent through the contact form for suggestions and advice during the development of this series.  This is a growing opportunity for everyone entering into product development as well as a great review for us "well seasoned" in the field.

It is recommended for those that have specific questions to use the BASIL Networks Contact Form for questions to separate them from getting lost in the general comments for each blog presentation.  For all specific design request or contracts please feel free to contact me.


Reference Links:

ITF Selected Components

MAX-II EPM1270T144C5  Pin Assignment Template

BOM Spreadsheet and Component datasheets ZIP file

PGA281 Programmable gain Amplifier Datasheet
IS66WVE4M16EBLL 64Mbit (4M x16) Pseudo SRAM Datasheet
Alliance Memory AS1C8M16PL 128Mbit (8Meg x16) Pseudo SRAM

FileStructure_LAD.h  - File structure for LAD Controller Software

Altera® Quartus Download 9.1 sp2 from Archives
Intel®/Altera® Quartus Lite 20.1 Download

Requirements Traceability Matrix  (RTM)
Project Management
Mezzanine Board

The majority of Internet scheme and protocol information are from a few open public information sources on the net, IETF (Internet Engineering Task Force) RFC's that explain details on the application of the protocols used for both IPv4 and IPv6 as well as experimental protocols for the next generation Internet and the Network Sorcery web site.  The remaining of this series on the IoT platform will be from BASIL Networks MDM (Modular Design Methodology) applied with the Socratic teaching method.   Thank You - expand your horizon- Sal Tuzzo

Network Sorcery: http://www.networksorcery.com
The Internet Engineering task Force: IETF - RFC references
Wikipedia https://en.wikipedia.org/wiki/Main_Page

Memory Segmentation
The Memory Management Unit (MMU)
Virtual Address Space
Virtual Addresses and Page Tables
Extended Memory


Previous Part 21 IoT Core Platform - Peripheral I/O Development - Peripheral Device Real World Testing -Continued(Mar  23rd, 2020)

IoT_Index


Publishing this series on a website or reprinting is authorized by displaying the following, including the hyperlink to BASIL Networks, PLLC either at the beginning or end of each part.
BASIL Networks, PLLC - Internet of Things (IoT) -Security, Privacy, Safety-The Information Playground Part-22 Peripheral I/O Design - Peripheral Devices- Real World Testing - (Mar 10, 2021)

For Website Link cut and paste this code:

<p><a href="https://www.basilnetworks.com/Blog/index.php?op=ViewArticle&articleId=37&blogId=1" target="_blank">BASIL Networks, PLLC - Internet of Things (IoT) -Security, Privacy, Safety-The Information Playground Part-22 Peripheral I/O Development:-<i>Real World Testing (Mar 10,  2021)</i></a></p>

 

alt

Sal (JT) Tuzzo - Founder CEO/CTO BASIL Networks, PLLC.
Sal may be contacted directly through this sites Contact Form or
through LinkedIn

23 Mar, 2020

Internet of Things (IoT) Security, Privacy, Safety -Platform Development Project Part-21

Part 21: IoT Core Platform Development;- Peripheral I/O Device Design
The IoT Embedded Core Platform -Peripheral Devices Real World Testing - Continued (Update May 14, 2020)

"The first rule of any technology used in a business is that automation applied to an efficient operation will magnify the efficiency.  The second is that automation applied to an inefficient operation will magnify the inefficiency." ( Bill Gates - Oct 28 1955 - )

As we stated in past presentations in this series development costs are easily exceeded when the performance expectations and in that case performance requirements are not documented properly, hence: the infamous "TBD" -To Be Determined.  This forces a direction change "AND" direction changes are commonly not listed as part of  performance expectations.  Sometimes the development "process" is faulty or just plain broken period.  The intent of this series in not to maintain the insanity of over budget development costs but to disrupt it in order to allow the creation of new habits that will give a solid foundation for engineering practices to be successful when starting a development project.

For the new designer that is taking on the learning of CPLD and FPGA design, "Each design completed is experience for the next lever of development."  There are only experiences and techniques that will bring you to the advanced level of applications.

IoT_Index Quick Links

Quick review to set the atmosphere for Part 21:
Every new year technology advancements exceed the previous years, we get to apply some new technologies released to gain experience.  This past year 2019 many new devices and IC emerged on the market for developers and applications for the IoT arena.  We will address some of the new devices and in fact may have an impact on our IoT Core development series.  This is the new reality of product development in a fast changing technology environment, sometimes the product becomes obsolete before you are able to get it to market.  Project management becomes a bit challenging when faced with obsolescence during development   Simple buy for the end of life becomes a marketing nightmare knowing that the competition will release a competitive product with the latest hardware while your product is using obsolete or discontinued components.  Designers and development integration becomes a real design dilemma determining if the replacement "If There Is One" will cost a complete redesign or a firmware/Software update and more.

So as of this writing here is what has been happening in the micro silicon embedded world,   A couple of new embedded processors have emerged from all the Mergers & Acquisitions and many processors have been discontinued.  Performance and features such as higher core speeds and multiple cores make it applicable to many AI (Artificial Intelligent) applications.  Several interface ICs have been released that may help the interface sensors etc. along, however careful consideration has to be taken to insure that the interface will be around for at least five years to accommodate the embedded lifecycle requirements.

It is easy to loose focus on the project with all the new technology being released so a quick review to remain focused on the series.  This series is focused on the development of a Core Platform IoT applications and has a dual purpose, first an educational project development for the entrepreneurial mindset along with applications, second the implementation of remote sensing and control incorporating safety, security and privacy over a wide range of peripherals including wireless.  This should be kept in mind since we will be designing in some redundancy for reliability and security.  For the yourg newcommer to product design and development if you are chosed to be mentored by a senior accept humbly, it will be the most rewarding experience of earned knowledge you will every have.  

Since the wireless industry and advanced since the introduction of the IoT devices we have also started a new series on Wireless technology in a separate category, Wave Phenomenon, Antennas & Radiation in the category of Wave Propagation and Antennas.

The ITF development Project:
The ITF did create a change in direction, however did not change the desired results, this is not only common in development projects but is some cases required to meet the desired expectations.  The issues here are handling multiple projects, resources available and the changed timeline, just a short change in direction to complete the objective.  A validation that the real time product development process is far from being a linear process from conception to finished product.

Remember changing direction overnight does not mean changing goals our the final destination, it is just a better way to insure you will reach the desired destination.  In my years of exposure to the design arena as a designer, troubleshooter and mentor I have had the honor of experiencing innovative and passionate creators to realize that the creative thought process of an individual is not a linear step function, 1, 2, 3 ...N, probably because humans are not robots, that follow a preprogrammed set of processes as some may think of engineers.  The innovation of the human mind subconsciously is always performing scenarios to find the best solution for the task at hand and "developing habits along the way".

We will pick up from the last couple of parts in the series that cover CPLD#1 and CPLD#2 to put us in perspective for a complex design.  In Part-19 the digital I/O section of the ITF which included CPLD #2 was presented.  Due to the complexity and size of the blog only the first part of the Digital I/O was presented which included the following sections.

In Part-20 the continuation of the digital I/O design which included the internals of CPLD#2 registers, preliminary flow diagram and pin requirements of the CPLD and the first pass CPLD#2 design.  The important part of this section was the templates for tracking and creating PCB footprints by pin number and pin names.

What we want to cover in Part 21:
In Part-16 through 20 we addressed the issue of why we should consider testing of the peripherals (Proof of Design) PoD with a prototype build to insure when we interconnect several of the peripherals the throughput required for the applications will be met.  This allows the opportunity to change the development direction from the peripheral point of view if performance expectations become an issue. Developing a test methodology that will be used for testing peripherals for the platform keeping in mind peripheral throughput limitations. These are new habits being developed at a conscience mind set level that will connect to become the default critical thought process during development.  With that stated we will continue moving forward with detailing the Interface Test Fixture (ITF) CPLD #2.

Updating the IPD project documentation for the ITF (Interface Test Fixture) to keep track of the development process, "Documentation is a living process during development", a good habit to make!  The reference for the ITF is the functional block diagram Figure 16.1

Lets Get Started:

Some questions answered from our readers
"What happen to the series?,
"It has been a few months since part 20 publication, are you going to continue the series?
To answer the above two questions - YES the series will continue.   BASIL Networks, PLLC own and maintain several on-line servers as well as off-line servers all located on-site. We are in the process of upgrading them to the latest and greatest compliant requirements aw with all upgrade and new installations this took "longer" than planned, HHMMmm I think I said that about product development timing some where, well again we have added another first hand experience to our portfolio. Cool   We also started a new blog design category as we stated several times in this series about wireless and security, this new blog is addressing the wireless section and as always we start at the very basics and work up to the expert level for our readers.  The new blog is Wireless, Radiation and Antennas.  Yes, I know it was mentioned and linked a few paragraphs back. It is the old poetic license of repeating ones self three times, so that leaves one more time somewhere in this part.

Refresher on Security,  CPU Code and Encryption:
When designing products that are controlled by some type of processor keeping a security mindset should always be an active part of the design process.  There are several sections of a security policy to be taken into consideration when designing any product that will be connected to a network be it global or local.   Access security implementation is by far the most time driven authentication process today.  From Single Password Authentication to Multi-Factor-Authentication software driven it takes time to access and perform all the decryption, table look-ups and more.  Most financial and many government sites have incorporated MFA and generally it is less expensive to incorporate a software change than to upgrade physical hardware.  A few of the top level questions to address for the security poplicy are:  Who will be accessing the device?,   What type(s) of encryption to use?,  Who owns the keys?,  Are the encryption keys changeable?,  are just the few top level ones.  We will cover a few ways of insuring security and privacy for the IoT Core Platform as we move forward.

OK, why the security refresher?  This is where we incorporate the mindset to design in access security, proivacy and safety for the I/O part of the IoT Platform.  What does this have to do with the Core IoT Platform?  Putting security in the ITF allows us to test different access methodologies for access control.  For the ITF the design incorporates the CPLDs internal FLASH that will allow the user to embed a series of encryption keys as well as ITF device serial number, and other identifying characteristics which may also be encrypted and assigned to a specific PC it is used with for added security.

It is much easier to incorporate hardware access security in the initial design stage than attempting to add a hardware security feature after the design is complete with some type of after thought patch.  The MAX-II  EPM1270T144C3 incorporates an 8192 bit FLASH that may be organized by byte or word and configured during the CPLD programming stage as 1024x8 or 512x16.  By enabling the security bit the FLASH and the CPLD configuration are not readable to the user, however the ability to completely erase the device remains, keeping the data from being read.  This is a good start, however we still have to create a security policy to implement into the CPLD FLASH.  This is addressed later in the series.

ITF CPLD#1 A/D vs CPLD#2 Data I/O Peripheral:
A brief comparison of the CPLD#1 A/D design and CPLD #2 High Speed Digital I/O Design.  Both incorporate a SRAM buffer to collect data at a periodic programmable rate or manually triggered externally.  However the design parameters are completely different, for one the high speed digital I/O requires a 10ns static RAM and the A/D is much slower 1us conversion time so a 1 typical PSRAM of 55ns is sufficient and much less costly.

A/D Channel vs Data I/O Throughput 
A quick comparison of the two technologies, the A/D Channel is a 1 us Sample rate via a 50MHz 16 bit  (20ns x16) Serial interface.  The actual conversion time is approximately 700 ns which allows ample time for the serial data to be transferred into the PSRAM on board without missing any points until the number of samples has been obtained.

The Parallel 16 bit interface in CPLD #2 is different since we require the full 50 MHz or faster x16 bit transfer rate without missing points.  This simply states that the full time cycle to write to memory is less than 20ns in order to meet the 50 MHz transfer rate.  The main difference is that the memory type for this requires a full Static RAM type of 10ns or faster verses a Pseudo SRAM type of speeds 70ns typical.  The faster the SRAM the higher the cost also the faster the throughput and the more power required which generates more heat.  There are a smaller selection to choose from when designing with 10ns or faster SRAM devices.  There is a list in the Reference Links of the selected high speed SRAM devices for this project at this time.

Peripheral vs. Functions
When designing any product including test fixtures the Peripheral vs Function discussion will always arise to insure the best price per performance is addressed.  With the technology advancing at such a rapid rate ICs are being release on a regular basis to eliminate support ICs as well as reduce design and testing time.  There are pros and cons with all designs and they have to be addressed.  The approach we are applying in the development of the ITF is flexibility and programmability and separating the front end sensing to separate IC(s).

Create a Preliminary Timing Diagram
As we stated before creating a preliminary timing diagram of the expected performance takes some practice and is always recommended for speeds that get close to the limits of the CPLD and will aid in troubleshooting timing propagation delays after programming.  Figure 21.0 is the preliminary timing diagram for CPLD #2 and as we see it would require capturing the data in a 5 ns window in order to be ready for processing in 10ns.

The other issue is as we review the pin count it appears that will not be room to incorporate the designs for the serial ports, hence, the SPI, I2C and QSPI types of protocols.  We could always incorporate a third CPLD, however it may be better to actually use the 16 bit interface and create the serial interface CPLD as an actual IoT Platform serial Interface then use the IFT to test the data transfer for the IoT Peripheral device itself.  Remember the IoT Platform I/O will remain a constant while the processors evolve. The expectations is to design a 10ns data throughput interface with the maximum data width possible.

alt
Figure 21.0   CPLD #2  Preliminary Timing Diagram

From the IDE functional block diagram in Part-20 and the Preliminary timing diagram above we should be able to easily create the simulated IDE logic timing diagram with all the delays and the DMA performance for CPLD #2.  The IDE release 19.1 Quartus Prime Lite Edition for the CPLD is available for download in the Reference section below.  We will use an older release 9.1sp1 for this design for the time being since we have not had the time to fully evaluate the new release and test the differences that have been completed to date. To keep this consistent we will recompile later after the design is complete and compare the differences in releases.  Prior to Intel/Altera M&A Altera several of the releases of Quartus Web incorporated sets of different CPLDs and selected FPGA and they recommended that you use the release that is required for the device of choice, however a lot has changed since Intel is now merging this with their product line.  We have to look at redesigning several in-house devices since they were discontinued that has caused some issues in the lab as many R&D labs have encountered.  

During the timing setup a few design issues came up that have to be addressed.  The Functional block diagram is correct, however the implementation of the design has limitations and will not meet the expected throughput.  So what is missing for the CPLD #2 design that should have been presented at this time.  Lets take a look at what preliminary design procedure requirements for CPLD #2.

We have the following completed

OK, what we have left out is the preliminary functions I/O Register Map assignments.   Why do we want to do this prior to the design, well, since we did not create it on a spreadsheet the programmer will have to figure this out, not a good idea.  What could possibly go wrong if we just do it later? The black hole cliche of all times.  So lets complete the process requirements, it is easy to just not use a bit in a register than it is to try and add one later.

There are many schools of thought on register implementation for peripherals and there are same implementations that make it extremely challenging for software to get around and are not recommended.  Some general guidelines of register implementation to follow that insures a hardware/software balance.:

Following these guidelines allows the software configuration to be grouped in a contained block of memory to be easily programmed and viewed.  The register sets for CPLD #2 are all eight bit (byte) registers to meet the pin count of the 144 pin CPLD and leave a few pins for modifications.  A good rule of thumb to follow is always try and stay below 80% of the available capacity of the device using to allow manual changes to the routing to address unique propagation delays..

Peripheral Control/Status Register = 8 Bits - 1 byte wide register

From the above list CPLD #2 requires 14 byte wide registers for setting up the peripherals functions as well as the 14 control lines to load each of these registers. This is a good place to start.  The front end of CPLD #2 for the Register set would require 5 bits to address the uo to 32 registers and control strobe lines combined to initiate the device manually and set the different mode configurations.

CPLD Simple Propagation Delay Test
In Part-14 we touched lightly on the internals of the CPLD connection matrix and the internal Logic Element blocks. This is one of the areas that many talented engineers are stilled challenged with and it is not going away either. It is the internal connection matrix that contributes to those propagation delays that make or break a design.  Just because a design works with the fastest speed device and you are below or within the lower speed device parameters that costs less does not mean the lower speed device will work the same way.  Connections matrix is a large contributor to propagation delays however complexity and available Logic Element gates and where they are between Logic Element blocks adds to the interconnection issues.  Here in the lab we have experienced this many times where we pushed the CPLD to 90% of resources and changed speeds and still had to redesign the CPLD to make it work.  Now take this understanding and think you can easily change manufacturers or CPLD series and expect it to perform the same or better without real world testing.  One of the areas we have found to be root cause of failure is with major bit shift switching from 111100001111.  The matrix in a CPLD are not neat and organized as if you would hand wire it, they are routed via an algorithm similar to auto route in a PCB and that can get cluttered.

The typical misconception with programmable logic devices from engineers and engineering managers that have not had the opportunity to actually design with them is "Oh, just make changes to the CPLD/FPGA and we do not have to run a new PCB", well folks that is not a good thought process to be transmitting.  As we will see below the same design acts completely different with just a change of the device speed and nothing else.  Propagation delays are variable depending on the complexity of the design.  Internal noise (crosstalk) with adjacent interconnection planes can cause false triggering and other types of spikes throughout the device.

Ok, now that we have created a preliminary timing diagram for CPLD#2 of the critical data flow of the memory data I/O it is time to encourage good design behavior when designing with CPLDs and FPGAs.  A good general practice when designing with CPLDs or FPGAs is to test the waters on speed and general throughput when attempting to push the limits.  Sometimes boolean logic just seams illogical when it comes to connecting points in a CPLD and FPGA matrix.  To Test the CPLD being used, a simple circuit was compiled and simulated to show the timing diagram of the functions shown in Figure 21,1 incorporating a simple pipeline binary decoder 1 bit to 2 line and 2bit to 4 line decoders with an gated clock.

IMAGE_CPLD_TEST
Figure 21.1   CPLD #2  Speed Test Function

The expected digital sequence for this design is shown in Figure 21.2 and is a basic binary decoder 2 and 4 line with a front end latch for one clock delay.  The front end latch is to insure that the decoder input is presented with the binary bits all at the same time in order to eliminate spikes that will cause other timing issues.  The GateEN is the standard D-Type F/F and it operates on the rising edge of the clock and is expected to respond in less than 1ns.  Figure 21.2 shows what is expected and the simulation results Figure 21.3 and 21.4 are two different results.  This is one of the reasons it is encouraged to create an expected digital timing diagram when ever possible prior to actually using the IDE to design the CPLD or FPGA.  Over the years we have found that the simulation process with IDE's does perform well, however being able to simulate all functions for the design is where many fall short and the problems show up in real world application after the product is on the market.

CPLD_2_Expected_Timing
Figure 21.2   CPLD #2  Pre-Timing Diagram Expected 

To start the highest speed CPLD available was selected for the simulation for the test design in Figure 21.1.  Figure 21.3 shows the simulated timing for the 144 pin device selected.  To understand the propagation effects of speed to configuration of the CPLD we will run the same timing simulation but with the slower device.  The CPLD The actual part for this simulation test is the EPM1270T144C3 for the fastest speed and EPM1270T144C5 for the slowest speed both handle speeds faster than 10ns.

CPLD-Test-SimTimingScaled.jpg
Figure 21.3   CPLD #2  Speed Test Simulated Timing Fastest Device (C3) 1270LE Timing

Observing the two timing diagrams we easily see that the device becomes unstable at the same frequency.  When we look at the pin to pin delays for the different speed devices C3 vs C5 we get 6.3ns and 10.1ns respectively.

CPLD_Test-SimC5Scaled.jpg
Figure 21.4   CPLD #2  Speed Test Simulated Timing Slow Device (C5) 1270LE

Keeping the same design just changing from a 1270 Logic Element device to a 570 element device Form Fit Function the results for the C3 devices are also very different as shown in Figure 21.5 below.  The MAX-II EPM1270T144C3 and the EPM570T144C3 are form fit pin interchangeable however the performance differ greatly with just a simple design configuration.  Timing changes with complexity and pin assignments as well.  When we look at the pin to pin delays for the different architecture 1270LE C3 vs 570LE C3 speed devices we get 6.3ns and 5.5ns respectively.

CPLD-Test-SimC5-570Scaled.jpg
Figure 21.5   CPLD #2  Speed Test Simulated Timing Fast Device (C3) 570LE

The question is what is the stable operating frequency for the devices.  We ran a series of test frequencies from 10ns to 1ns clocks  and the high speed device was selected as well as adding some delay functions in the clock line.  When you get below 10 ns it is very typical that you will run into propagation delay issues when as the available LEs (Logic Elements) of the CPLD become part of the design.  The experience we have seen here in the lab is every unique interconnecting trace adds about 0.4ns delay so if you have to go through 3 unique interconnects before you get to the pin you already exceeded the clock time of 1.5 ns for a 3ns clock cycle as shown in the timing diagrams.  Looking at Figure 21.4 we see that the propagation delays for DOUT2 exceed the clock and the output is completely missing.   Also DOUTA and DOUTB overlap which would cause other timing issues if DOUTB must start after DOUTA ends.  OK, now think of the fact that manufacturers make design updates to these devices at random. This makes interchangeability a very risky business for critical reliability devices.

There are ways to compartmentalize the design and keep the I/O ports for the section close to the LE and not split them between several LEs.  After thought additions is where the problems arise the most since the addition may have to reconfigure the CPLD differentially to accommodate the functionality designed in and the device becomes unstable.  The in-depth analysis of CPLDs and FPGAs go beyond the scope of this blog series however, it is important to present some of the challenges attributed to CPLD design when designing peripherals since the majority of designs today incorporate some from of programmable logic.  As a refresher on why the 144 pin CPLD was selected, mainly because it has been around for many years, designed into many devices with no end of life in sight at this time.  The MAX II and the MAX V are 2 different pins, MAX-II has 116 I/O pins and MAX-V has 114 pins.  We have not tested the performance or Form Fit Function physically in the lab since all our designs use the MAX-II and the devices are still readily available for all levels of designs.   Although the selected device is more costly the choice for same performance the MAX-II EPM1270T144C3 is the one we will use for this design. If this were going into a large volume product a separate PCB attached to the ITF could be used to AQL (Acceptable Quality Limit) evaluation of devices per lot.

Register & Bit Assignments
The correct design sequence is to create the CPLD #2 register map first, then create the bit assignments for each register prior to the actual design of the CPDL.  Yes, we did this backwards not just to show the time it requires to undo the design then implement the new updates, but in this case we attempted to try and reuse some of the assignments from the reuse design library.  The reuse design was taken from another project that involved a memory buffer.  Well the implement of the reuse parts and adding some functions that are required for this design just added more delays which  from several years experience with timing delays it would be better to just redesign the CPLD from scratch..  This is typical of many reuse scenarios and will now cost more time to undo and redo the design, this is our learning curve on using a design that are close to what we wanted but not exact.  Also when we canalized the propagation delays the throughput transfer speed also fell short of the expectations.  

The Register and Bit Assignments will be presented in Part-22 of the series along with spreadsheet templates.

How to Obtain a Finished ITF:
Our plans when the ITF is finished, is to offer an ITF to the public that has many more features than the one being developed for this presentation.  We already have completed two different PoD products to get ready for manufacturing and offering custom development for contract manufacturing companies and the entrepreneur small company that want to setup a development test base for future and present development contracts.  Please use the BASIL Networks Contact Form to be put on a mailing list when we are ready to supply the manufacturing prints if you are interested in purchasing the entire system manufactured and tested.

[Selection_Menu]

SUMMARY:
OK, this is a lot to present in a single part as with most of the parts of this series as we dive deeper into the designing devices.  The reuse of similar designs has it pros and cons, in this case it was a con since the timing of the design caused the redesign of the interface.  In many cases modifying a design ends up taking longer and incorporates less functionally that would have been to have the flexibility and freedom to do the design from scratch.  Many discussions with senior engineers and innovators have confirmed this through the years.  The difficulty lies with those that make the decision for the engineer and have not been in the design field for some time.  There is no disrespect meant with that statement, it is just a fact of human nature that when you are away from the details for a while you only focus on the results and not the process to get there; and by the way, as it should be, however when the facts are put on the table those managers (leaders) trust those that are at the detailed level and follow that lead and tend to managing the project.  This behavior of trusting the front line knowledge has been a proven process for a successful project development.  So, why would you have high level knowledgeable individuals on a field of interest drawing a salary and not listen to them? Cool that works the majority of the time and is what a responsible team is not only required but is how a team is expected to perform. It has been said by many of lecturers through the industry from different fields "Engineers design and build cities, Managers run them after they are built".  Ok, end of lecture to encourage engineering and managerial behavior

Updating and Modifying CPLDs / FPGAs
Here in the lab we have been very fortunate until just recently that our internally designed and developed test fixtures have stood the test of time for almost 20 year.  We still have some XP systems still running mainly due to third party PCI cards that will not run in any of our Windows 10 x64 Enterprise systems with PCI slots.  Some of our internally developed test equipment that incorporate discontinued CPLDs have to be redesigned.  When we looked at the replacement cost of some COTS instrumentation the lab requirements were just not available in a single product as well as the price being out of the expected budget.  The best approach is still to design the test fixtures to meet our needs with components that are anticipated to be around several years.

As shown in the above in Figures 21.1.through 21.5 FFF (Form, Fit, Function) does work but not exactly the same propagation delays add a high risk for critical equipment. This FFF discontinuity is also effective by revision changes in the IC itself.  Several programmable logic IC manufacturers have pin-to-pin replacement for their series of CPLDs, however think of a revision change in one of the devices that are labeled FFF in the series and is changed out for more LE's  or to add to that a different manufacturer and a new PCB layout.  We all get the picture now; so what is the solution.?  Building a physical prototype and test, test and test again for all functionality.   

Over the years the major contributions to the timing and propagation issues were designs focused for the selected CPLD / FPGA series that approached the maximum resources available in the device. This contributed to major carry glitches usually four bit and above blocks  111100001111 along with the data path within the interconnecting matrix for the chip.  As we see the FFF factor sort of falls from the rooftop of a tall building when upgrading to the next pin interchangeable device in the series.  What we also be noticed is using actual I/O pins as test points in the simulation since they are also routed through the connections matrix, when removed from the final design the device also performs different.  Many engineers use internal pin assigned names that are simulated at the point inside the chip that are picked which will change the timing as well.  The big pro of programmable logic devices is that you can change the design of a device and use the same PCB.  A major con of programmable logic devices is that the propagation times change as the density changes for the application and compensation gates have to be added to insure stability and functionality of the device.

This update to the ITF section adds a single channel DMA controller that is shared as a Digital Logic Analyse for monitoring CPU BUS timing.  This becomes a very useful reuse design since not all designs will survive the reuse environment as we mentioned previously, only about 5% will be a true total Plug'N'Play reuse.  Experience has shown that FPGA and CPLD designs that incorporate the simplest of modifications have the risk of reduced performance, it is the nature of the beast.  We selected the larger of the MAX-II Logic Units to allow optimization and future additions if required.

Quartus 9.1 to Prime Lite 19.1 Timing simulation comparison:
The timing test was run on Release 9.1 and 19.1 the results were different for the same tests.  Release 19.1 has better routing optimization and also allows both functional and timing simulation that concur with both devices in the same family and speeds.  This is a good thing since we will be updating to release 19.1 when we update the custom equipment in the lab.  These are important issues for all in house equipment since we have talk to several labs some still running XP and are forced to remain on older versions.  An update will be presented as we continue to test and replace obsolete CPLD and FPGA devices

CPLD Register Identification
Since we neglected this because of the Reuse factor the thought was since the design matrix was already defined prior all that would have to be done is reassigning the bits for each register.  Well renaming the registers and bits did not turn out as expected and, oh by the way to make it clear; "It Very Seldom Does work as expected."  So, re-mapping the registers is required, that ok, however the complexities became clear that re-mapping lead to changing functions sequences and that lead to other changes and it soon became obvious when we started modifying the design the 50% mark of changes were present and the end is not clear.  At that point purely by experience it was decided to just start from scratch and layout the CPLD to fit the application with a few spare I/O pins for future growth.  In this reuse case it was fortunate that the redesign was recognized early before many man hours of troubleshooting and development were lost.

Final Note
Changing poor engineering habits are difficult however not impossible to correct.   Humans are very flexible they all have the ability of learning anything with applied effort, the only impasse is the mind set that if negative will defeat any attempt to grow and instill fear of learning.  The key is to acknowledge the initial behavior, no it will not change overnight - it took a while to become rooted.  Bringing the development behavior to the surface and acknowledging the behavior is the first step in this series to bring the development process to a winning level.   The expectations of this and other series on the blog is to encourage the behavioral changes as well as present a project that is useful for many applications.  Behavioral modifications is a personal and private process that takes time and requires trust within ones self.  

As the series progresses the author, Sal Tuzzo will be available for discussion through the BASIL Networks Contact Form for those that want to apply this series to conduct their own experiments.  I will always be appreciative for the private comments sent through the contact form for suggestions and advice during the development of this series.  This is a growing opportunity for everyone entering into product development as well as a great review for us "well seasoned" in the field to just refresh our human DRAM (Dynamic Random Access Memory).

It is recommended for those that have specific questions to use the BASIL Networks Contact Form for questions to separate them from getting lost in the general comments for each blog presentation.  For all specific design request or contracts please feel free to contact me.


Reference Links:

ITF Selected Components

MAX-II EPM1270T144C5  Pin Assignment Template

BOM Spreadsheet and Component datasheets ZIP file

PGA281 Programmable gain Amplifier Datasheet
IS66WVE4M16EBLL 64Mbit (4M x16) Pseudo SRAM Datasheet
Alliance Memory AS1C8M16PL 128Mbit (8Meg x16) Pseudo SRAM

Intel®/Altera® Quartus Download 9.1 sp2 from Archives
Intel®/Altera® Quartus Lite 19.x Download

Requirements Traceability Matrix  (RTM)
Project Management
Mezzanine Board

The majority of Internet scheme and protocol information are from a few open public information sources on the net, IETF (Internet Engineering Task Force) RFC's that explain details on the application of the protocols used for both IPv4 and IPv6 as well as experimental protocols for the next generation Internet and the Network Sorcery web site.  The remaining of this series on the IoT platform will be from BASIL Networks MDM (Modular Design Methodology) applied with the Socratic teaching method.   Thank You - expand your horizon- Sal Tuzzo

Network Sorcery: http://www.networksorcery.com
The Internet Engineering task Force: IETF - RFC references
Wikipedia https://en.wikipedia.org/wiki/Main_Page

Memory Segmentation
The Memory Management Unit (MMU)
Virtual Address Space
Virtual Addresses and Page Tables
Extended Memory


Previous Part 20 IoT Core Platform - Peripheral I/O Development - Peripheral Device Real World Testing -Continued(Nov  3rd, 2019)

 

IoT-Index


Publishing this series on a website or reprinting is authorized by displaying the following, including the hyperlink to BASIL Networks, PLLC either at the beginning or end of each part.
BASIL Networks, PLLC - Internet of Things (IoT) -Security, Privacy, Safety-The Information Playground Part-21 Peripheral I/O Design - Peripheral Devices- Real World Testing - (Mar 24, 2020)

For Website Link cut and paste this code:

<p><a href="https://www.basilnetworks.com/Blog/index.php?op=ViewArticle&articleId=28&blogId=1" target="_blank">BASIL Networks, PLLC - Internet of Things (IoT) -Security, Privacy, Safety-The Information Playground Part-21 Peripheral I/O Development:-<i>Real World Testing (Mar 24, &nbsp;2020)</i></a></p>

 

alt

Sal (JT) Tuzzo - Founder CEO/CTO BASIL Networks, PLLC.
Sal may be contacted directly through this sites Contact Form or
through LinkedIn


1 2 3 4 5 6 7 8 9 10 11  Next»
Copyright© 1990-2019 BASIL Networks, PLLC. All rights reserved
webmaster