| Andre's profileFurtaSpace - www.afurtad...PhotosBlogLists | Help |
|
November 09 Software Engineering Research and ValidationDoing Software Engineering research requires well thought out planning and validation approaches. Studying such topics, I have come across a couple of interesting papers. A summary of their most interesting points (for me) follows below. Writing Good Software Engineering Research Papers · A research paper should include a number of questions stating your contribution (question you answered, why the reader should care larger questions addressed), your new result (previous work , new knowledge, details) and why readers should believe you (evaluation standard/method, evidence connecting claim and result). · Types of software engineering research questions include: method/means of development, method for analysis or evaluation, design/evaluation/analysis of a particular instance, generalization or characterization, feasibility study or exploration. · Program committees look for a clear statement on the specific problem you solved and how it will scale. Tell your answer and why it matters. · Quantitative models have a better success rate (as being recognized by the scientific community) when compared to quantitative models. Tools and notations are well represented, usually as auxiliary results in combination with a procedure or technique. · Types of software engineering research results include: procedure or technique (should be really a procedure, not advice or guidelines), qualitative or descriptive model (e.g., a taxonomy, a framework, well-organized interesting observations), empirical model, analytic model, tool or notation, specific solution/prototype/answer/judgment, report. · You should explain your result in such a way that someone else could use your ideas. Precisely tell what’s new: the idea, the application of the idea, the implementation, the analysis or what? · For tools: was the contribution the technique that is embedded in the tool, the effectiveness of the tool when compared to others, the applicability of the tool or what? Does the tool simply support the main contribution, or is itself a principal contribution? Can the idea be applied without the tool? · Explain how you build atop previous work, otherwise you may damage credibility if the committee can’t tell whether you know about related work. · A paper that simply reports on using numerous elements together is not enough, even if it’s well engineered. There must be an idea or lesson or model readers can take and apply to other situation. · Types of software engineering validations include: analysis, evaluation, experience, example, persuasion, blatant assertion. · Analysis, actual experience in the field and good use of realistic examples tend to be the most effective ways of showing why your result should be believed. Careful narrative, qualitative analysis can also work. · Pilot studies that lay the groundwork for controlled experiments are often publishable by themselves. · If you claim to improve on prior art, compare your result objectively to prior art. · If you performed a controlled experiment, explain the experimental design: hypothesis, treatment, what is being controlled, collected data, how it was analyzed, significance of results, confounding factor, whether the conclusions follows rigorously from the experimental data. · For PhD thesis validation section: global statements (“always, fully”) often require analytic validation, qualified statements (“X% of improvement”) can often be validated by evaluation or careful examination of experience existential statements (“we found an instance of”) can sometimes be validated by a single positive example. Students can restate thesis claims to reflect more precisely what the theses actually achieve. · Suggested abstract structure: two or three sentences about the current state of the art identifying a problem, one or two sentences on the contribution, one or two sentences about the specific result and its main idea, one sentence about how the result is demonstrated or defended. Selecting Empirical Methods for Software Engineering Research · Five classes of research methods are considered to be the most relevant to software engineering: controlled experiments, case studies, survey research, ethnographies and action research. The choice of methods depends upon the theoretical position (stance) of the researcher, access to resources and how closely the method aligns with the questions that have been posed. · The selection of methods for a given research project depends on many local contingencies, including available resources, access to subjects, opportunity to control the variables of interest and the skills of the researcher. · One of the first steps in choosing an appropriate research method is to clarify the research question. · The most obvious question is not the best choice for a starting point. · Defining the precise meaning of terms is a crucial part of empirical research and is closely tied with the idea of developing or selecting an appropriate theory. · In the early stages of a research program, we usually need to ask exploratory questions to understand the phenomena: existence questions (Does X exist?), description and classification questions (What is X like? What is it for? How to categorize/measure it?), descriptive-comparative questions (How does X differ from Y?) · Then we need to ask base-rate questions about the normal patterns of occurrence of the phenomena: frequency and distribution questions (How often does X happen? What is the average amount?), descriptive-process questions (How does X normally occur? What is the process by which it happens? What is its sequence of events?), relationship questions (Are X and Y related?) · Then we should ask causality questions and, finally, design questions (How to achieve X?) · Theories support the process of empirical induction because an individual study can never offer conclusive results. Each study adds more evidence for or against the prepositions of the theory. Without the theory, we have no way of making sense of the accumulation of empirical results. · A controlled experiment is an investigation of a testable hypothesis where one or more independent variables are manipulated to measure their effect on one or more dependent variables. Each combination of variables is a treatment (configuration). · A precondition of conducting an experiment is a clear hypothesis. The theory also helps to decide who the subjects are and what the tasks should be. · A case study is something very different than just worked example. It can be exploratory (initial investigation to derive hypothesis and build theories) or confirmatory (to test existing theories). · Case study research is most appropriate for cases where the reductionism of controlled experiments is inappropriate. · The major weakness of case studies is that the data collection and analysis is more open to interpretation and researcher bias. For this reason, an explicit framework is needed for selecting cases and collecting data. · A survey research is used to identify the characteristics of a broad population of individuals. It can be done through questionnaires for data collection, structured interviews or data logging techniques. · Sampling bias causes problems in generalizing the survey results, because the respondents to the survey may not be representative of the target population. Low response rates increase the risk of bias. · Ethnography is a form of research focusing on the sociology of meaning through field observation. The goal is to study a community of people to understand how members of that community make sense of their social interactions. · In action research, the researchers attempt to solve a real-world problem while simultaneously studying the experience of solving the problem. There’s lots of debate on the validity of action research as an empirical method. · Mixed methods research recognizes all methods have limitations and the weaknesses of one method can be compensated for by the strengths in other methods. · Criteria for validity (for positivists/reductionists) include: construct validity (whether theoretical constructs are interpreted and measured correctly), internal validity (whether the results really follow from the data), external validity (whether claims for the generality of the results are justified) and reliability (whether the study yields the same results if other researchers replicate it). · In reporting positivist empirical studies, it is important to include a section on threats to validity, in which potential weaknesses in the study design as well as attempts to mitigate these threats are discussed in terms of the criteria above. · Strengths for improving validity (from a constructivist point of view) include: triangulation (use different sources and methods), member checking, rich/thick descriptions, clarify bias, report discrepant information, prolonged contact with participants, peer debriefing, external auditor. · It is important to document the original planned research protocol, and all subsequent deviations to it, to allow other researchers to understand the study design, interpret the search results and replicate the study. · Methods that are primarily qualitative include ethnography, case study and action research. Quantitative include controlled experiments and survey research. · With regard to research and validation, it is important to plan ahead, for smooth analysis and interpretation of results. Qualitative Methods in Empirical Studies of Software Engineering · In software engineering, the blend of technical and human behavioral aspects lends itself to combining qualitative and quantitative methods in order to take advantage of the strengths of both. · Advantage of qualitative methods: they force the researched to delve into the complexity of the problem rather than abstract it away. The results are richer but labor-intensive. · Participant observation and interviews are one of the most common data collection methods. Interviews can be structured or unstructured (in the latter case, interviewees are the source of both questions and answers). · “Coding” is the process of extracting vales for quantitative variables from qualitative data in order to perform statistical/quantitative analysis. This approach is especially difficult when the concept to be coded is subjective in nature, when the terminology used to describe it varies and is difficult to interpret, and when different data sources disagree. · The distinction between qualitative and quantitative data has to do with how the information represented, not whether it is subjective or objective. · The “constant comparison methods” is a data analysis method that tags qualitative data with terms, and then groups related information together to look for relationships. · Cross-case Analysis a data analysis method that cross-compiles data. · A hypothesis cannot be proven; it can only be supported or refuted. This is true independent from the usage of qualitative or quantitative methods. · One of the most important ways to help confirm a qualitatively generated proposition is to ensure the validity of the methods used to generate it. Some validation concerns are: representativeness, possibility of researcher effects on the study, triangulation, anomalies in the data, negative case analysis, replication and member checking. · Researchers have a marketing role as well, trying to promote the importance and usefulness of the empirical study in software engineering. · Empirical studies can be classified in: blocked subproject-project (multiple projects and subjects to reduce bias), replicated project study (same project, multiple subjects), multiproject variation (single subject, projects before/after some treatment), single project study (single project, single subject, similar to a case study, possibly compared to some organization baseline). BR, October 26 PM verb blitzWhile positions in the Dev/Test discipline at Microsoft are straightforward to understand for newcomers, this is not exactly true for the PM (program management) discipline. It's not the same thing as project management and relies much more on a technical side.
Steven Sinofsky already detailed what program managers do at Microsoft. My contribution here is a "verb blitz": what are common actions performed by PMs? Here you have a list compiled from some colleagues:
BR, -- AFurtado October 20 How I extend Visual Studio?These are the projects of individuals/companies who presented in the Microsoft's Development Tools Ecosystem Summit 2009 how they are extending Visual Studio:
I didn't speak this time, but you are always invited to try out my SharpLudus project which extends VS by combining model-driven development with XNA, hopefully enabling a more productive game development process. BR, Keyboard key mapping – solving the ASUS question mark (?) problemJust sharing something useful: I have an ASUS laptop with a brazilian keyboard but the question mark/slash key (? and /) never worked with the brazilian ABNT layout. After browsing the web I found out that such a key was being mapped to the Control Key. The way to solve this was to map it to the question mark key, by setting the following registry key:
This tells your computer to map the right control key to the “/?” key. Log off, then log on. The problem will be solved. BR, September 24 Singleton design pattern vs. static classesDuring an internal .NET 101 training at Microsoft one of my students asked a very interesting question: why would we choose to use a Singleton design pattern instead of simply using a "global" static class?
After some thinking and investigation, here are my conclusions of the benefits for sticking with Singletons:
BR, July 08 Brazil wins Imagine Cup for the 5th time! / Brazil vence a Imagine Cup pela quinta vez!versão em Português mais abaixo For the fifth consecutive year, Brazil wins a category at the Imagine Cup world finals (the biggest student technology competition in the world). This time, team LEVV It, from Pernambuco, got the first place with in the Game Development invitational with the game Choice. As one of the judges of the XNA Challenge Brazil 2009, I was able to play the game and was impressed by its game design: arcade action is well dosed with strategic insights, and the result is lots of fun! Brazil also got fantastic results in other categories, such as first place in Design, first place in Interoperability and third place in Software Design (main category), among others. Click here to check all the results. Other relevant news: Romania got the big prize (first place in Software Design) and Imagine Cup 2010 will be held in Poland (cool). Being a “retired” contestant starting with Imagine Cup Japan 2005, I’m really glad to know that Brazil’s results at Imagine Cup are not one-off, but very consistent across the years. Congratulations to the Brazilian group at Egypt! BR, ------------------------------------------------------------------------------------------------------------ Pelo quinto ano consecutivo, o Brazil vence uma final de Imagine Cup, a Copa do Mundo da Computação. Dessa vez, o time LEVV It, de Pernambuco, conseguiu o primeiro lugar na categoria de Desenvolvimento de Jogos com o jogo Choice. Sendo um juiz do XNA Challenge Brazil 2009, eu pude jogar o jogo e fiquei impressionado com o seu game design: ação estilo arcade se combina muito bem com estratégia, resultando em muita diversão. O Brasil também conseguiu resultados fantásticos em outras categorias, como os primeiros lugares em Design e Interoperability, além de terceiro em Projeto de Software (categoria principal), e outros. Clique aqui para ver todos os resultados. Outras notícias relevantes: a Romênia ficou com o grande prêmio (primeiro lugar em Projeto de Software) e a Imagine Cup 2010 será na Polônia. Como um competidor “aposentado” da Imagine Cup, começando pelo Japão em 2005, estou realmente feliz em ver que os resultados do Brasil na Imagine Cup não são por acaso, mas sim muito consistentes ao longo dos anos. Parabéns, comitiva brasileira no Egito! []s June 28 Twitter reminderDid you know that:
Those are a sample of tips (plus personal thoughts, opinions and news) I’m sharing on twitter: www.twitter.com/awfurtado. See you there. BR, June 26 Finding bugs through exploratory testing in WolframAlpha[This post is part of a series. To see the previous, click here. To understand the context, click here] The Bing Other release that had some attention in the search area was WolframAlpha. As you might know, while you can use Bing or Google to do some simple computations such as measurements conversions and calculator operations, WolframAlpha is a “computational engine” by default (i.e., it computes your input towards a result rather than searching for web pages). WolframAlpha is cool, giving interesting data as results that Bing and Google do not provide (yet?). Try “Seattle to Recife” there and you will understand what I’m talking about. For example, it can show nice mortgage graphs based on your input (loan amount, interest per year, loan period, etc.). Check it out: As any other Software Development Engineer in Test, I tried running a couple of “unexpected” queries to check out the application behavior. What about a negative value for the total number of years for my loan period? There you go: Effective interest rate is now screwed. Graphs and tables in such a scenario are not good either: data get collapsed to the center of the graph, or $Fail! Since one of the coolest features in WolframAlpha is to combine multiple computations (such as “Weather in Seattle when Obama was born”), let’s go a step beyond and assign the square root of -1 as the total number of years for my loan period: The loan period is now the imaginary number (“i”)! Isn’t it awesome? Only your imagination can tell now when you’ll be done with your mortgage! :) The computation of graphs and other stuff now takes a while and fail as a timeout. Export to PDF feature is also broken. Yay, we found another bug! Fortunately those are minor errors which won’t avoid me from keeping using WolframAlpha. In fact, I’d love to see Bing moving (actually extending) into such a direction, i.e., computational engines. And what about enabling people to add their own computational logic, formulas and conversions to the engine? I mean, creating a computational engine leveraged by the power of Web 2.0? For example, extending Wolfram or Bing towards a universal slang translator? PS: Using imaginary numbers to explore other interesting WolframAlpha scenarios is left as an exercise to the reader. :) BR, The statements or testimonies I offer in this post represent my own personal views. June 22 DSLs: the bad and the uglyThe panel “DSLs: The Good, the Bad and the Ugly” that happened at the OOPSLA 2008 conference discussed the advantages, disadvantages and challenges from the industry and academia on the use of domain-specific languages (DSLs). While the advantages brought by the panelists were somewhat expected (more productivity, more abstraction, etc.), the disadvantages and challenges were quite interesting and not that obvious. Here you can find a compilation of them:
BR, June 01 [old] Microsoft FrontPage Server ExtensionsProblem when deploying a ASP.NET app from Visual Studio: Unable to create the Web site ‘XYZ’. The Web server does not appear to have FrontPage Server Extensions installed. Solution: http://www.rtr.com/fpse/ BR, May 28 VSX 2010VSX (Visual Studio Extensibility) 2010 news follow below. The WPF-empowered extensibility is inspiring… Are you building new tools to simplify your life or change the way people work? Check out the opportunities to create innovative extensions for Visual Studio 2010.
Examples of cool things you can do:
· Overlay rich metadata on top of the editor to bring new information to your fingertips · Develop new project templates to accelerate your work (examples: New Project Dialog) · Create innovative user interface elements using the WPF Shell · Upload your extension to the Visual Studio Gallery (http://visualstudiogallery.com) to find it in the Extension Manager Other things you should know: · With v2 of the Visual Studio Gallery that went live on 5/14, you can use the Extension Manager to find products from the ecosystem from within Visual Studio *(with Beta1, the Extension Manager will only surface extensions posted to the Gallery in the VSIX file format; Beta2 will include the MSI format) · We just released the new SDK along with the new Shell redistributables, look for them at the dev center: http://msdn.com/vsx · If you’re developing extensions for VS 2010, you’ll want to stay tuned to the dev center for more information
Fun things to help you get started – these can be found in the Extension Manager (Tools>Extension Manager): search for Editor Extensions
FAQ:
What is the Extension Manager and how do I find it?
The Extension Manager in Visual Studio enables developers to discover Visual Studio Extensions from within the product itself. It consumes data directly from the Visual Studio Gallery, giving developers direct access to partner and community products that extend and enhance Visual Studio. If you have a free or free trial version of your product hosted on the Visual Studio Gallery, a developer will be able to download and install that product directly from within the IDE. (In Beta1, the in-product experience will show only extensions uploaded as a VSIX. In Beta 2 we will expand the in-product experience to also include extensions uploaded as MSI and reference links to other download sites.) The Extension Manager enables a developer to install, upgrade, uninstall, enable, and disable extensions, making it easier to manage a portfolio of extensions. To find the Extension Manager: Tools>Extension Manager. So this means the Visual Studio Gallery can now host downloads?
Yes, we’re excited to announce that the Visual Studio Gallery can now host your downloads. Keep in mind that at this time the gallery does not include a commerce framework so we recommend creating a free or free trial version of your product that can be hosted on the Visual Studio Gallery. Of course we still support reference links for users who prefer to host their own downloads. What else is new @ the Visual Studio Gallery?
Thanks to some great work by the MSDN team we’ve moved the Visual Studio Gallery to be part of MSDN. Some of the benefits: · The new navigation model supports all types of Visual Studio Extensions and has sections for Tools, Controls, and Templates · The integration with MSDN now gives users a chance to take advantage of a shared profile across all MSDN properties. Contributing to the gallery can enhance your MSDN reputation giving high quality contributors the recognition they deserve. · Visual Studio Gallery contributions offer an optional discussion board to separate your reviews and give contributors an opportunity to post updates, respond to feedback, & discuss bugs. · All entries include a usage chart to give contributors a sense of which of their products are getting the most traffic on the Visual Studio Gallery.
What’s the deal with the VSIX file format I’ve heard about?
Both the new Visual Studio Gallery and the Extension Manager work with .VSIX files, which is a new deployment model for extensions, similar to a renamed .ZIP file. This transition provides Visual Studio users a new low-overhead, reliable way to deploy Visual Studio Extensions. There’s tooling available to help you with this, and you can find instructions and other helpful tips at the Visual Studio Extensibility Dev Center: http://msdn.com/vsx. Questions? Email catherine.crim@microsoft.com BR, May 02 Infected… not by swine flu, but by the Silicon Valley entrepreneurship spiritLast Friday I had the opportunity to present at the Microsoft Imagine Cup Innovation Accelerator Workshop, in Mountain View, CA. The presentation was related to insights and experiences, as a former Imagine Cup competitor and short-experienced entrepreneur, on what make profitable business and innovative people. The audience was the Imagine Cup 2008 winning teams, who are now changing their focus from the prototypes created during the competition to the establishment of real business aimed at the market. The best moment of the day was at the final presentations the teams did to a board of real angel investors and venture capitalists. I was able to learn a lot from the investors and VCs, in the context of startups looking for capital to start their business. The major takeaways were:
BR, The statements or testimonies I offer in this post represent my own personal views. April 25 Send your name to Mars through NASA / Mande seu nome para Marte pela NASAThis link enables you to add your name (along with thousand others, of course) to a microchip that will be sent to Mars. What for? No idea. Probably to market NASA’s expeditions and improve how society perceives them. ----- Este link permite que você adicione seu nome (junto a milhares de outros, claro) a um microchip que será enviado a Marte. Para quê? Não faço idéia. Provavelmente para fazer um marketing das expedições da NASA e melhorar a percepção deles pela sociedade. BR, April 22 Brazil from America’s eye – random thoughtsThis article from Newsweek touches with a couple of new insights the outworn topic “Brazil: the country of the future”, which I (should I say every Brazilian?) got to know since the very first moment I was able to understand Portuguese. I’d like to discuss a recurrent thing. It’s said there, in the article: “Brazil's power derives not from guns but from its immense store of natural resources, including oil and gas, metals, soybeans and beef—and it has become a key supplier of markets in Asia and closer to home.” If we have handful natural resources in a peaceful environment, why aren’t the benefits of this advantage enough to invest and harvest the most important resource ever: people? It’s ironic our culture is so well-know for being creative, but at the same time “Brazil’s power” does not derive from entrepreneurship, technology and creativity. For instance, the job of the dreams for the (majority of?) average Brazilian student is to apply for, and be approved in, public initiative (government) positions, which has an awesome cost-benefit (worked hours X wages earned). Stability wins; taking risks to make a change, doesn’t. The article can praise the Brazilian government and its external politics, but there’s still too much internal homework to be done, especially in the most impactful social area: base education, which is the ultimate investment a nation can do to spin-off the poor towards enough knowledge to enable autonomy and independence from any short-term economical incentives. And, of course, to create the entrepreneurs of the (country of the) future. BR, The statements or testimonies I offer in this post represent my own personal views. April 11 [echo] I’m a PC and I’m 4 and a halfIn short: a cute little asian(-ish) girl has much more expressive marketing power than Jerry Seinfeld…
It seems MS is finally finding its way through the ads. The I’m a PC campaign is receiving good feedback, specially its last reincarnation, “The Laptop Hunters”.
BR, April 05 The day my laptop exploded - outcomesAbout seven months ago I blogged about a laptop of mine whose brand new battery simply exploded. It caught on fire and made my old Toshiba A75 to become a chemical weapon. Last week, seven months later (better late than never!), I got in my mail a $500 check from the battery manufacturer. That should be enough to cover the 4 year old laptop, the new acquired battery and costs for shipping the laptop back to the manufacturer. Happy ending? I’d say so, if it didn’t involve taking risks to injure one’s body parts and put one’s house on fire. BTW, do you backup your stuff regularly? BR, April 03 Managing permissions for “Everyone” in SharePointJust documenting something useful here: I was wondering what is the SharePoint equivalent for Active Directory’s “Everyone” users, so that I could apply permissions for a SharePoint list or survey to all users of my intranet. The answer seems to be: NT AUTHORITY\Authenticated Users. BR, March 22 Essential facts about digital games industry – 2008 report / Fatos essenciais sobre a indústria de jogos digitais – relatório de 2008[versão em português abaixo] I just came through the Entertainment Software Association (ESA) Essential Facts 2008 report for digital games. I believe it was released months ago; anyway, a couple of things caught my eyes:
Meanwhile, the news tell that the game industry growth has reduced but not stopped, even with the current economical crisis. BR, ---------------------------------------------------------------------------------------------------------- Esbarrei com a edição de 2008 do relatório de fatos essenciais de jogos digitais da Entertainment Software Association (ESA). Acho que ele foi lançado há meses atrás; mesmo assim, algumas coisas me chamaram a atenção:
Enquanto isso, as notícias dizem que o crescimento da a indústria de jogos diminuiu mas não parou, mesmo com a crise econômica atual. []s March 20 [echo] 3D Immersive Xbox games? / Jogos Xbox imersivos 3D?[versão em português abaixo] Speaking about gestures, something interesting was reported here. It’s suspected that Microsoft might acquire a technology to enable immersive 3D games, without the need of any device at all. It seems some envisioned scenarios for the future of game development are coming faster than expected… PS: hope that becomes available to XNA game developers as well! ----------------------- Falando sobre gestures, algo interessante foi reportado aqui. Suspeita-se que a Microsoft está para adquirir uma tecnologia que permite uma imersão 3D em jogos, sem a necessidade de qualquer dispositivo. Parece que alguns cenários vislumbrados para o futuro do desenvolvimento de jogos estão chegando mais rápido do que o esperado… PS: espero que esteja disponível também para desenvolvedores XNA! BR, March 13 [echo] Unveiling the sixth sense / Revelando o sexto sentido(versão em Português abaixo) Since Minority Report, the concept of “gestures” has taken UIs to the next level. While gestures are already enabled today in software by frameworks such as WPF, the presentation below shows we’re not too far away from turning such a thing into more integrated day-by-day experiences, such as taking a picture.
Desde o Minitory Report, o conceito de “gestures” tem levado interfaces para o próximo nível. Se por um lado frameworks como o WPF já nos permitem aplicar gestures em software, a apresentação acima indica que não estamos tão longe de transformar isso em experiências mais integradas do nosso dia-a-dia, como tirar uma foto, por exemplo. BR, |
|
|