CAPTAIN SIM FORUM
General >> Hangar talks >> How the bugs got past testing?
https://www.captainsim.org/forum/csf.pl?num=1207625315

Message started by tasev on Apr 8th, 2008 at 3:28am

Title: How the bugs got past testing?
Post by tasev on Apr 8th, 2008 at 3:28am
I would like to understand a bit more about BETA testing.  What is it all about and what are the objectives?  I currently understand that it is a testing process intended to help find faults and weed out bugs - is that correct?

I am puzzled as to why, after 3 Beta tests, there are - as some here imply - 'blatant issues' with the product.  Sure there are always service packs post release, many times more than one; but in most cases that I have seen, these fix either minor issues, or problems only people with experience and a sharp eye would notice, or even enhancements.

But some of what I have read here are so operationally obvious, it makes me wonder how they got past testing?  I can only imagine how much pressure we here at the forum must have put on you to get a finished product.  

I must honestly admit, myself being a veteran purchaser of your products and others, am quite surprised.

Title: Re: Questions on BETA testing
Post by Captain_Sim on Apr 8th, 2008 at 3:43pm
Thank you.
Well, the 3 Beta tests are nothing comparing to hundreds of in-house tests completed by our team members and the type rated pilots.

So, how the bugs got past testing?
Actually it is not about the testers' "sharp eye". It's not a big deal to find 100 bugs, but to fix them without breaking other things.  

For example the '757 Captain' system programming consist of 1600+ print pages of handwritten C++ and XLM code. Where every parameter in every line is interconnected with hundreds of others. It's just programing.
Add thousands of handmade bitmatps, hundreds of textures, tens of 3D modles (thousands of polys each), hundreds of animations, flight model, hundreds of sounds. Now imagine those MANY THOUSANDS of items must look\feel\sound real and work in accordance with flight manual. IT'S A HUGE JOB.  

Same software for a real aircraft costs hundreds of thousands dollars.  ::)

As for the customers' "pressure" (we call it feedback), it helps to set a priority for what to fix first, last or ignore.


Why other developer made this\that system\feature right?
1. All developers are different, all products are different but nothing is perfect. All products (hamburgers, software or cars) have strong and weak points.  
2. We keep working on our products improvement.  


Title: Re: Questions on BETA testing
Post by tasev on Apr 9th, 2008 at 2:15am
Thanks CS.  I think this will help some of the novices understand what it takes to go through such a project, and why they still see problems with the sim.  The last thing I'd like to see is people making wrong judgements about you guys.

Title: Re: How the bugs got past testing?
Post by signmanbob on Apr 18th, 2008 at 4:19am
Captain Sim, thank you for linking your above statement to some other feedback within the forum.  I think that your explanation is so good that it is a shame that it can't be linked to the support forums of every major complex sim developer, because I believe they are all experiencing the same short sighted misunderstanding from their consumers.

I have always been extremely impressed with what I consider to be the artistic audio, visual and interactive beauty that is expressed in a complex accurate simulation like the 757 Block F.  I have always wondered about and suspected that there must be a herculean amount of work and frustrations that go into completing a product that replicates to a satisfactory degree (not to mention the high degree of the 757 Block F), the most complex piece of machinery in existence for the general public to enjoy at such an affordable price.

Your explanation has enlightened me on this as, I'm sure, it has many others.  I think it should be added that these simulations not only have to strive for stableness, accuracy and realism, but also have to interact with a hodgepodge of other add-on programs which many times, are running in multiples along side the sim.  Real airline software doesn't even have to put up with weather simulators, passenger simulators, voice simulators, checklist simulators, moving map flight planners, scenery enhancements, etc. running within it's calculations.

Title: Re: How the bugs got past testing?
Post by Captain_Sim on Apr 22nd, 2008 at 1:59pm
Thank you for understanding!

Title: Re: How the bugs got past testing?
Post by aladdict on Jan 6th, 2011 at 3:51pm
I wish to add more questions to the Bata testing. I wish to ask why did you choose the 707 model you did??

All the hype that surrounded it was as if it was the model Pan Am flew for its first flight in 1958 but it isn’t. The first Pan Am flight was a 707-320A model with turbo jet water injection engines. The model you released wasn’t available till 1962 with the fan engines. Although you did replicate the interior PSU’s as the 1958 model but the exterior is definitely the later model. Why didn’t you release the original model with the original engines?? It would look better with that early Pan Am livery. If you do plan to release the early 707 with the turbojet engines I will definitely buy it.

Please don’t get me wrong I really like your model and plan to buy any later or retro versions you release. O yes there is one other thing I found incorrect, The 707 never had a light mounted on the nose gear. The 707 had so many lights it really didn’t need a nose gear light. It was equipped with 4 landing lights, two inboard and two outboard retractable lights along with two inboard runway turnoff lights mounted with the inboard landing lights. The 727 was the first to have a light mounted on the nose gear because its outboard landing lights were mounted in the flower flaps and were usually retracted except during takeoff and landings.  


Title: Re: How the bugs got past testing?
Post by TurbofanDude on Jan 18th, 2011 at 4:24pm
I think Captain Sim really hit the nail on the head. I'm a payware designer, and I'm working on a scenery. I noticed that the airport in the default scenery is about 100 feet off from the real world position, so therefore, ultimate terrain users will have issues with compatibility. To fix that, I would have to re-make the whole airport to move it. Is that worth it? I think that with the amount of coding Captain Sim has to do, they run into this a lot. Is it worth it? So there may be one or two glitches, but like they said, NOTHING is perfect.

Hope I helped to explain a bit too

Collin Biedenkapp

Title: Re: How the bugs got past testing?
Post by pipat4 on Aug 16th, 2011 at 12:46pm
Thanks for any new information.

Title: Re: How the bugs got past testing?
Post by popoing on Sep 5th, 2011 at 6:46am
thanks you ;D

Title: Re: How the bugs got past testing?
Post by pj747 on Sep 16th, 2011 at 2:12am
Looks like that guy is a future ex-member

Title: Re: How the bugs got past testing?
Post by boeing247 on Sep 16th, 2011 at 4:23am
Shoot. We had gone so long without a spammer. At least it's not one of those bots that posts multiple, nonsensical ads.

Title: Re: How the bugs got past testing?
Post by CoolP on Sep 16th, 2011 at 4:28am
You mean I shouldn't buy earrings from him?  :o
Couldn't you have told me before?!  >:(

Title: Re: How the bugs got past testing?
Post by boeing247 on Sep 16th, 2011 at 4:30am

CoolP wrote on Sep 16th, 2011 at 4:28am:
You mean I shouldn't buy earrings from him?  :o
Couldn't you have told me before?!  >:(


Hey, maybe it's not spam--perhaps he's selling Flight Sim earrings.  ;D Check your FO for any interesting new apparel on your next flight.

Title: Re: How the bugs got past testing?
Post by Captain Sim 2 on Sep 16th, 2011 at 2:18pm
He is more than ex-member now - he is banned ex-member :)

CAPTAIN SIM FORUM » Powered by YaBB 2.6.0!
YaBB Forum Software © 2000-2024. All Rights Reserved.