Home > Development, Flight Simulation, Flightsim, FSX, Technical, Vancouver > Vancouver+ for FSX status and test plan overview

Vancouver+ for FSX status and test plan overview

Sunday, May 20th, 2007

Holger, Scot and I are currently testing/refining the configurator/installer.  Once we are satisfied with it, we will release the first beta to the test team in the next few days.  Testers have been chosen, the testing forum is set up, test plan written, and test allocations done.  Testers have signed up to check the product by geographic sector, and by functional area.  While the team is testing, I’ll finish off the CYNJ buildings and some other loose ends.

Our approach is to provide the beta team with a fairly well tested release.  We’ll fine out how it works on different platforms, including some older machines that we’ve included in the test group.  We will minimize the number of beta drops (test releases) by running a set of regression tests prior to each drop.

For those interested in a deeper view into our testing procedure, here’s our simple test plan.  Note that is directed to the beta team only, and is provided here should it be useful for other developers.

Vancouver + FSX – Test Plan and Sector Map

Because of the large scope of this project, both in terms of geographic area and features, we will be asking folks to sign up to do specific types of testing. This way we will help ensure nothing falls through the cracks.

There will be 7 geographic test regions, Alpha through Golf, and several non-geographic functional tests.

We also will be doing a simple form of regression testing: when we specify a new beta release is an RT beta, all tests must be run again. This minimizes the chances that fixes and enhancements do not cause collateral damage. We will of course work to minimize the number of complete test runs we do. For those who are interested, I have a document available entitled, “Regression Testing for Flight Simulator Addons”.

Of course, we still encourage mucking about, flying at will, and just having fun. That may find problems we haven’t thought of.

Here are the geographic test sectors (see below):

Image

Some notes on reporting:

• please use screenshots with coordinates enabled (ctl-z), limited to 125kb, 6 per post (though multiple posts are possible), and (please!) max. width of 800 pixels. If you don’t want to hand-edit your screenshot use something like Grab-Clip-Save http://www.simviation.com/fsutilities_gen3.htm
• please report in the appropriate sectional threads (see below) and avoid starting new threads unless it’s a very general (or very specific) issue
• report time of day/season
• for AI traffic monitoring, for those of who have FSX Deluxe, use the Traffic Toolbox
• please ensure you have looked over existing bug reports carefully

Reference Material

• maps for British Columbia: http://toporama.cits.rncan.gc.ca/toporama_en.html or here http://webmap.em.gov.bc.ca/mapplace/minpot/general.cfm (the second link requires a free plug-in for the interactive map). http://maps.gov.bc.ca/ and click on “Land and Resource Data Warehouse Catalog”
• googleEarth http://earth.google.com/
• VanMap, for detailed downtown airphotos: http://www.city.vancouver.bc.ca/vanmap/o/orthophotos.htm
• Burnaby, for detailed Burnaby airphotos: http://webmap.city.burnaby.bc.ca/publicmap/viewer.htm
• Richmond GIS for detailed Richmond airphotos: http://map.city.richmond.bc.ca/website/gis/viewer.htm
• Photos: http://www.webshots.com/search?new=1&source=mdocsheader&words=
• Other photos: search at http://www.google.com/imghp
• Oblique air photos: http://www.globalairphotos.com/
• Buildings: http://www.skyscraperpage.com/ and http://www.emporis.com/en/

Known Issues – please don’t report unless noted below:
• List coming

Test Section Overview

Notes on Test sections:

• Please report issues in the thread provided
• Delta and Echo areas have many features, it is suggested that if you sign up for one of these, you don’t sign up for other areas
• you can report issues outside your chosen area, but please use the correct thread

Sign up by section:

• Section 1a – Geographic Area Alpha
• Section 1b – Geographic Area Bravo
• Section 1c – Geographic Area Charlie
• Section 1d – Geographic Area Delta
• Section 1e – Geographic Area Echo
• Section 1f – Geographic Area Foxtrot
• Section 1g – Geographic Area Golf
• Section 2 – AI
• Section 3a – Airports – land-based
• Section 3b – Airports – Seaplane bases
• Section 3c – Airports – Heliports
• Section 4 – Third party compatibility

Everybody will do:

• Section 5 – Performance
• Section 6 – Installer/Configurator

Test Section Descriptions and What to Look for

All Sections

Please report any crashes and shutdowns with as much detail as possible so that others can try to recreate the situation.

Please check for and report major frame rate drops and increased loading times.

Suspension of disbelief: Holger’s own measure of success is to fly around and not come across places where the environment looks or feels “wrong”. For example, are specific textures (rocks, forests, water) strange or of the wrong color? Do the shorelines look off? Are the settlements placed well? Anything that makes you go “hmmm…” or “ouch!”

Section 1 – Geographic Tests – by regions (Alpha – Golf)
Sign up by geographic area – See test regions above.

Objects (Buildings, dams, marinas, bridges, other structures) – look for flickering, incomplete or inaccurate textures, blend with context, misplacement

Photoscenery – look for blend with context, edges

Lake elevations and river placement and elevations – look for plateaus, pits, breaks
mistakes in vector line designations – look for roads as streams and vice versa
Roads and railroads – look for misplacement, breaks, river/lake crossings
Shorelines – look for missing, incursions: river meets or leaves lake
Utility Cuts – look for how lovely they look
Landclass – look for mistakes in landclass (eg desert instead of forest)
Autogen building encroachment – look for buildings on steep mountainsides or other inappropriate places

Testing:
– for your chosen region(s) fly a pattern and check the above in detail
– fly a coarse pattern over the region in each season (Winter, Spring, Summer, Fall) and each time of day (Dawn, Day, Dusk, Night)
– report by lat/long with screenshot with coordinate display turned on (ctrl-z)
– you may be asked more than once to repeat the full test before final release

Section 2 – AI
Seaplanes – look for smooth departure from dock or ramp, and smooth docking or parking. Smooth traffic flow at runways and taxiways.
Small boats – look for collisions in paths, crossing land, etc.
Cruise ships – look for collisions in paths, crossing land, etc.
Helicopters – look for collisions with buildings, land
Testing: use Traffic Explorer to monitor takeoffs, landings and progress.

Section 3 – Airports
Testing: Check ATC, runway designations, parking,

Section 4 – Third party compatibility
Test with any third party addons you have and report anomalies

Section 5 – Performance
We will provide some test flight scenarios and required settings to create some benchmarks.

Section 6 – Installer/Configurator
Install and uninstall, reinstall, repair.

Appendix – Test Regions (see map above)

Test Region (TR) Boundary
Alpha (Howe Sound) 50° 37.50’N 123° 45’W x 49° 30’N 122° 58’W
Bravo (Garibaldi) 50° 37.50’N 122° 58’W x 49° 30’N 121° 55’W
Charlie (Fraser Canyon) 50° 37.50’N 121° 55’W x 49° 30’N 120° 56.25’W
Delta (Vancouver) 49° 30’N 123° 45’W x 49° 11’N 122 °58’W
Echo (Golden Ears) 49° 30’N 122° 58’W x 49° 11’N 121 °55’W
Foxtrot (Hope) 49° 30’N 121° 55’W x 48° 52.03’N 120 °56.25’W
(includes part of US, test that too)
Golf (Fraser Valley) 49° 11’N 123° 14’W x 48° 52.03’N 121 °55’W
(includes part of US, test that too)

Advertisements
  1. Alwyn
    Monday, May 21st, 2007 at 10:17 am

    It is grezt to see that this is getting so close: really looking forward to it!

    Quick question: right from the start in FSX I have had a “reef” running from Stanley Park in a southwesterly direction across the entrance to English Bay/False Creek. I use FSGenesis’s terrain mesh and landclass. I was hoping that SP1 would have cured this issue but it is still there.

    Questions: a) any known cure for this?
    b) Any chance that it will be addressed in the Vancouver+ package?

    Keep up the great work!

  2. Monday, May 21st, 2007 at 10:23 am

    Great to hear!

    Yes, that’s a known issue in FSX.

    a) the sure is to provide a revised file that corrects this.
    b) Yes we fix that and many other oddities.

  3. Alwyn
    Monday, May 21st, 2007 at 10:40 am

    Excellent, and thanks for the prompt response.

    I am tremendously impressed with the commitment and enthusiasm that I have seen from “Vancouver+ musketerers” (yourself, Holger, Scot etc.) and Justin at FSG in doing this labour of love and still finding the time to respond to all kinds of questions you must have heard a thousand times before.

    Once the FSX Vancouver+ arrives I can finally put FS9 (plus Vancouver+ of course!) out to pasture.

    Just as an aside: I have an AMD 3800+ (AM2, single core), eVGA 8800GTS & 2GB of DDR2-800 and I get quite acceptable framerates (consistently @ 20) with quite high level of detail on scenery but with low traffic. I do a lot of really low level flying, so this is satisfying for me.

    Given the SP1 dual core improvements and the CPU effect on FSX performance, I am seriously thinking of a X2 5600+ but perhaps I should wait for the new Barcelona CPU’s.

    Decisions, decisions….

  1. No trackbacks yet.
Comments are closed.