Author: wcarbona

Publishing Research with Non-U.S. Data

Q: As a non-US scholar, it’s often difficult to “sell” your research on countries other than the U.S. to U.S. scholars & journals. What kind of advice would you have in getting non-U.S. research in U.S. journals?

Florencia_torcheFlorencia Torche, New York University

Learning from countries other than the US is essential to broaden our understanding of educational dynamics. While we may all agree with that idea, nevertheless research on other countries may, at first sight, seem arcane to a US reader. It is also true that a large proportion of scholarship published in US journals is about US settings. However, research about other countries has increased its visibility and legitimacy in sociology over the past few years. My sense is that editors, reviewers, and the research community value it and are willing to engage it, under certain conditions. These conditions actually apply to all scholarship, but they are even more relevant when the setting is less familiar.

The first condition is the ability to explain why this study is relevant beyond the specific case being analyzed – to explain, early and forcefully, what general phenomenon your analysis illuminates. So if you write, say, about the determinants of shadow education in Korea (or in the US, for that matter) or the consequences of educational privatization in Chile (or in Cleveland, for that matter) your task as a researcher is to explain what we can learn in general about this phenomenon and its impact on attainment and stratification.

At the same time, your readers will want to know about the specific characteristics of the setting that you are analyzing. This description, I’ve found, is more effective when it first discusses what makes the case generalizable to others, and then describes what makes it unique. You should explain the contextual factors that distinguish your setting from others, and also those that are shared across settings. This second condition again applies to every piece of scholarship, but it is more explicitly relevant when your work examines a context other than the US.

These conditions imply striking a fine balance. You want to offer generalizable hypotheses and implications while retaining –indeed, drawing on— the particularities of the case under analysis in comparative perspective. This is challenging but extremely valuable. How would we be able to understand the relevance of the institutional context for educational outcomes and processes if we restrict our approach to the U.S. or any particular country? As much variation as there is inside this country, examining other countries and educational systems broadens both empirical understanding and theoretical approaches.

One temptation when studying contexts other than the US is to overemphasize the generalizability of your case, and the extent to which it illuminates a wide range of phenomena –all forms of shadow education, all experiences of educational privatization, and so on. Try to resist this temptation. Experts know the particularities of different cases, and will judge your overgeneralization as reflecting a knack for hyperbole (if they are benevolent) or ignorance (if they are not). A strong contribution is one that acknowledges the “limits of generalizability” and uses such limits to discuss sources of variation across contexts.

One question I sometimes get is whether it is convenient to explain “why this research is relevant for the US” when you do research in another country. This strategy is a bit US-centered, of course, and it may force you to engage discussions that are not a direct component of your intellectual agenda. In my view, to the extent that you consider the conditions above –frame your research broadly and engage both particularities and generalizability of your case thoughtfully—there is nothing wrong and there is much to be gained by explicitly offering implications for the US.

Finally, a word on practical matters: If English is not your first language; this does not give you a free pass. Your word choice should be accurate and your grammar should be perfect. You may want to have a native speaker edit your work. If you cite work in some other language, translate the titles so that your readers understand the substantive content. All this takes time and effort, but professionalism requires no less.

In sum, I believe good work about settings other than the US is not only increasingly accepted but increasingly embraced, as long as it makes a strong contribution to understand general class of social and educational phenomena. There is probably more effort involved in this type of work because it is not enough to address a policy or a change that is a “big deal” in the US. But perhaps that is a blessing in disguise, as it will force you to focus on the broader implications of your study, to the benefit of the entire research community.

Advertisements

Being a Good Discussant

The Hidden Curriculum is now well-rested, and ready to resume our normal activities. With ASA almost upon us, we thought it would be good to return with a conference-related post about how to be a good discussant. 

riegle-crumb_cAuthor: Catherine Riegle-Crumb, University of Texas

Q: I was just asked to be a discussant on a paper. What is the best/correct format? How does one critique a senior scholar (as an advanced graduate student)?

Being asked to be a discussant is flattering, and sure, it can be a little intimidating if your status is junior to the presenters on the panel. But the folks organizing the session were surely aware of that and picked you for a reason. Critiquing a senior scholar’s work is really no different than critiquing the work of a peer at a parallel career stage—basically, you want to be collegial and respectful in your delivery, and most importantly, have something constructive and relevant to contribute to the discussion.  That is, after all, your role as a discussant! Below are a few quick suggestions:

#1: Read all the papers in your session –carefully—and not the night before. This is more than a lot of more senior folks who serve as discussants actually do- so, you are already off to a great start!

#2: Your comments should be focused on helping the authors see what they might be missing, not in playing a game of academic “gotcha”. Perhaps the authors are telling a story that is broad in scope but could be more informative by going into more depth (e.g. talking about broad racial/ethnic trends in educational trajectories but not considering how this could vary by gender). Or, their argument could be strengthened by pulling on a body of literature that they have not yet considered. Or, they are making substantive or empirical assumptions that are not (currently at least) adequately supported or explained. In other words, don’t get hung up on details (did they make sure to try at least 3 different ways of imputing missing data?). Reserve your comments to issues that are related to the paper’s overall effectiveness. You can always communicate the “small stuff” to authors after the session is over, either in person, or in an email.

#3: Don’t spend too much time talking about how all of the papers “fit together” or how they are all related to each other. If you find some interesting parallels or contrasts among specific papers, you should of course mention those points in your comments. However, don’t go overboard and spend ten minutes talking about “where the subfield is headed.” Your main priority is to give constructive feedback to the authors – and also make sure that there is time left for questions from the audience!

#4: Remember: being a discussant is really not about you—i.e. don’t spend your time talking about your own research. This happens sometimes with more senior discussants who seem to forget that they are not in fact there to present their own work. Only mention your own research if it is HIGHLY relevant—and then do so VERY briefly.

#5: Finally, try to relax. Sure, there may be a few senior scholars who have forgotten that graduate students can give excellent comments. (You are, after all, being trained to read critically.).  But don’t worry about offending them. The reality is that they will likely be ignoring you anyway. JK! But seriously, most scholars do appreciate a careful reading and a constructive discussion of their work. If you do that, all of the presenters will surely be grateful for your efforts.

More Rapid Fire Q&A

cropped-hidden-curriculum-sm.pngQ: I’m starting a tenure track position at an R1 university in the fall. I’ve been interested in doing this bootcamp [name of organization withheld] for new faculty. Is a program like this is a good investment of time/money for new faculty? I have $10,000 in start-up funds. Should I use some of my start-up money on a program like this? Should I ask my Dean for additional funding to do this? I am a woman of color and I don’t want to start off seeming like I need extra coaching (but I feel like I do!).

First, congratulations on the new position! Best wishes as you move ahead into the next stage of your career.

Regarding your question, the HC does not want to endorse or promote any specific organization. So – we will provide a bit of general advice. Being successful as a professor at an R1 institution requires that you set long-term goals in terms of productivity. However, you must also have a viable short-term strategy for achieving those long-term objectives. “Bootcamps” and “writers’ groups/workshop” can be helpful because they provide a structure that compels you to meet short-term goals that will help you achieve your long-term objectives. They also provide both the accountability and support that comes with group membership.

If you think that membership in such a group would be helpful to you, by all means, go for it! Try it for a year, and if you don’t think it is useful, you can simply opt out. If you can demonstrate (after a few years) that membership in this organization has made you a more productive scholar, then you should ask your Dean to invest in your continued success by providing some additional resources for membership in future years. This approach should allay your concerns about seeming like you need “extra coaching” from day one. Alternatively, you could your ask your Dean to invest in an institutional membership so that other scholars at your institution can enjoy access to the same resources that have helped you be productive!

Q: I’m a grad student in sociology, particularly in economic sociology. I know that publications are valued . . . but how much does it matter? If I have a job market paper (or another paper) that the evaluating department thinks is a great paper, definitely publishable, is it same as having a published paper?  . . . . [A]re all papers, whether published or not, purely judged based on their merit?

It is true that research departments rely heavily upon publications as a screen in hiring. Publications help reduce uncertainty in hiring because past success is typically the best predictor of future success. Some job applicants submit great not-yet-published papers that are overlooked by hiring committees, which may seem unfair. However, it is important to recognize that publication decisions are largely driven by the judgments of reviewers who are specialists within subfields. If you publish an economic sociology paper in AJS, a department can reasonably infer that experts in economic sociology found your paper to be rigorous and important. Those of us who are not economic sociologists (i.e., the majority of faculty on most hiring committees) have a much more difficult time judging the quality of your unpublished work, and that’s why publications are such a useful signal in hiring.

Q: WHAT IS THE SIGNIFICANCE OF HIDDEN CURRICULUM?

In education research, the “hidden curriculum” refers to the implicit lessons about social life that children learn in school. While the “manifest curriculum” focuses on tangible skills and specific academic content (e.g., the 3R’s), the “hidden curriculum” teaches students about society’s norm, roles, and expectations. Bowles and Gintis famously argued that the hidden curriculum of schools corresponds to the capitalist order: children who are destined for work as white collar professionals learn different lessons about social roles and expectations than children who enter the working class.

We chose “The Hidden Curriculum” as the name for our blog because we believe that many of the most important lessons about having a successful academic career are never formally taught in graduate school. Indeed, we think the “hidden curriculum” of academic life is much more opaque than K-12 schooling because there is so much that is invisible to newcomers and outside of their direct experience. Advising and mentoring is incredibly important in socializing and “professionalizing” young scholars, but it is wildly uneven both between and within departments. Our goal is to level the playing field, and ensure that aspiring graduate students and junior faculty who read our blog have equal access to and knowledge of the “unwritten rules” required to be successful. Sadly, some people will fail in this business, but no one should fail because they didn’t master the hidden curriculum.

Preparing for the Non-Academic Job Market: Part I

We have received numerous questions regarding how to prepare for the non-academic job market. In the next few weeks, we will have several posts written by people who have varying levels of experience in this area. The first post is by Megan Shoji, a graduate student at the University of Wisconsin who was “on” the non-academic job market this year. Megan will begin working for Mathematica Policy Research after completing her PhD. this spring. 

shoji-megan2 Author: Megan Shoji, University of Wisconsin

Alt-academic jobs are attractive and viable for many PhDs, but as a student, it can be hard to know how to prepare. While there’s no single approach, I offer some general insights from my experience on the market this year.

An important first step is to identify why you’re interested in jobs outside academia, and what kinds of jobs interest you. This shapes both your job search and preparation, so ask yourself as early and regularly as possible, but ideally at least one year before going on the market.

Why do you want to pursue non-academic jobs? People often talk about life circumstances that may restrict success on the academic market (i.e., “I’m not sure I can get a faculty job”). This may be legitimate, for example if you’re limited by geography, but how you view a position can affect your chances of employment. It’s difficult to disguise intrinsic motivation, so if you see working outside academia as a fallback, you may be sunk before you begin — or at minimum you’ll lack the sincere enthusiasm that makes any candidate more attractive. To enhance your odds on the market (and personal fulfillment in your work), it’s important to find genuine reasons why a position you’re pursuing is good for you. Figure out what kind of work you want to do, how you want to do it, for what reasons, and in what kind of environment. In my case, non-academic jobs are ideal for what I want: to conduct research collaboratively, in an interdisciplinary environment, and toward the goal of directly impacting people’s lived experiences. But whether it’s that the job allows you to be in a city you love, to spend more time with your kids, or your partner pursuing his/her dream career, setting yourself up for success outside academia begins with recognizing how alt-academic opportunities are a good fit for you.

What kind of non-academic jobs do you want to pursue? Although people discuss ‘the non-academic market,’ realize that you’re really considering non-academic markets, differentiated by organization/job type and hierarchies. The work, environments, goals, and desired candidate skills can vary widely across markets. To know how to prepare, you must first get a handle on what’s out there and what interests you. Personally, I narrowed my search to research positions at policy analysis companies, state departments of education, and urban public school districts (particularly those committed to enhancing evidence-based decision-making and serving populations of focus in my research). From there, I generated a list of promising organizations to research. To find your interests:

Consult job boards listing non-academic positions, e.g. those run by the American Economic Association, Chronicle of Higher Education (https://chroniclevitae.com/), Policy Innovators in Education (http://pie-network.org/), or USAJobs (https://www.usajobs.gov/). Also conduct targeted searches by city or via organization websites.

Talk with as many people as possible about opportunities outside academia. As a grad student, discussing this with faculty mentors can be uncomfortable, particularly if they’ve made it clear that they prefer/hope/expect you to pursue faculty jobs. However, it’s likely in your best interest to begin this conversation as soon as possible. Many faculty have contacts outside academia are at least familiar with key nationally-renowned organizations. In my experience, while some said they believe academia offers incomparable advantages (or that it was their preferred choice for me), they also made it clear that their ultimate goal is that I find a productive and fulfilling career, and they were supportive and instrumental in my pursuit of alt-academic positions. Do your best to be forthcoming while remaining respectful and thoughtful toward the advice and feedback you receive. Also keep in mind that our hesitancy is sometimes rooted less in rejection from mentors and more in our own internalization of a culture that defines success as getting a faculty job.

Plan ahead and think strategically. Whatever your reasons for pursuing alt-academic jobs, don’t make the mistake of assuming it will be easy to get one, especially the one you really want.[1] Maximizing your chances requires time, actively tailoring your graduate education, and (often) additional training beyond program requirements. Once you know what you’ll pursue, use these guidelines for how to prepare.[2]

Don’t just dabble. Really cultivate skills, knowledge, and experience relevant for your target jobs. If strong quantitative skills are expected, instead of just taking a few extra classes, consider completing a minor, prelim, training program, or master’s degree in statistics. When attending conferences, joining professional associations, applying for awards, publishing, and developing your dissertation, writing samples, and job-talk presentation(s), make your activities relevant to non-academic organizations. Remember that skills that are desired but not required to get academic jobs—like the ability to communicate inter-disciplinarily or with practitioners and policymakers, work collaboratively, or balance multiple projects simultaneously—may be crucial to obtaining employment and succeeding in your ideal non-academic job.

Don’t just have it, show it. A potential employer won’t know you’re a great fit unless your application effectively communicates it. You could develop relevant skills through extensive extra coursework, but will your CV signal this as clearly as a minor or master’s degree would? And remember: while some non-academic organizations may prioritize publishing less than what’s typical in academia, many still rely on writing sample(s) and publication record to assess research skills.

Don’t be shy! Pursue contacts and mentors. Social networks are at least equally as important (maybe even more important) outside academia as within the academy. Networks not only help you access information and job leads but can also serve a vetting function in the hiring process. Make it a priority to set up informational meetings with non-academic organizations. Establish contacts locally, and make the most of research conferences—search programs and contact organizations to ask about popular conferences or whether anyone will be at those you’re attending. Pursue mentorship through summer or longer-term fellowships/internships, regularly offered at various non-academic organizations (and also great for gauging interest in an organization or career path).

 

[1] See point #1 in this guide to non-faculty job searching: http://chronicle.com/article/The-PhDs-Guide-to-a/143715/

[2] Also see: http://theprofessorisin.com/2014/02/26/questioning-your-future-in-academia-do-this-now-jackson-1/

Publishing “Non-findings”

Q: What should one do with papers that largely consist of non-findings? Is it just a waste of time trying publish a study without significant effects? Or, should I simply aim for lower quality journals with such papers?

Author: Bill Carbonaro, University of Notre Dame03ff41bd-56c6-4ea4-9fc1-5ef9ac0830d4

This is an important question that we must all confront as researchers. Not every study works out as we anticipated. No one wants to waste time on a project that has little chance of success. However, I think it is a myth that good journals only publish papers with statistically significant results.  Yes, there is a bias in favor of publishing papers with statistically significant findings, but there are many great examples of papers published in top journals that are built around “non-findings.” Here are the lessons that I have learned from these papers:

Characteristics of successful papers with “non-findings”

1. Have a good theory. For starters, it is important to have a well-grounded theoretical explanation for why one would either expect, or not expect, to find a relationship between the variables in your analysis. If your paper doesn’t find a relationship between X and Y, and the reader doesn’t have a clear idea of why we would expect that X should (or should not) covary with Y, then your paper is doomed. Ainsworth-Darnell & Downey (AD&D)(ASR, 1998) relied upon Ogbu’s much cited “oppositional culture” framework to generate several hypotheses about black-white differences in attitudes, behaviors, and school performance. They didn’t find any support for the Ogbu’s hypotheses, but that’s what makes the paper so interesting (and one reason the paper is so often cited!).

2. Cover new ground. The best case scenario is that one has a theory that has not been tested, and there is a “research vacuum” to exploit. AD&D covered new ground by being the first paper to systematically evaluate Ogbu’s claims with a multivariate analysis of nationally representative data. Their “non-findings” filled an important void in the field. In separate studies, Hallinan & Kubitschek  and I examined sector differences in learning among elementary school students, and found either no effects, or, positive public school effects. This went against the grain of prior research on the Catholic school advantage in high school (see point 3 below), but it also exploited the “research vacuum” in this area.

3. Re-visit old ground. Non-findings are especially interesting when they go against the grain of prior research. If prior research consistently finds that X matters for Y, and you find that X doesn’t matter for Y, that’s interesting!  The big question is of course WHY you didn’t find that X matters for Y. If you used different data than other scholars, you need to show why your data are superior in quality to prior research. If your data are more recent, then you have a very good story to tell: X used to matter for Y, but it no longer does. Of course, you need a good explanation for why X no long matters for Y. Think of Wilson’s The Declining Significance of Race and The Truly Disadvantaged. Much debated works, but they are compelling and provocative narratives! Finally, along the lines of points 4 and 5 below, you have an interesting paper if you can show that, in contrast with prior research, your use of higher quality measures and/or more rigorous methods indicates that X doesn’t matter for Y.

4. Have impeccable measures. If your analysis has poor measures of your key concepts, it shouldn’t be surprising that you have insignificant findings in your analyses. In order for a “non-findings” paper to be compelling, the reader should be convinced that the measures are credible, with high validity and reliability. This is a common problem in “non-findings” papers that I reject as a reviewer. AD&D used multiple measures of “oppositional culture” in their study, and while one might prefer better measures, their measures were credible enough to make a significant contribution to the field.

5. Have a rigorous (and possibly novel) research design. Happily, the bar of what counts as “rigorous research” is constantly being raised. “Non-findings” are especially important when one can make a convincing argument that prior research findings are in fact “artifacts” of weak research design/methodology.   For example, Guo and VanWey (ASR, 1999) used fixed effects models (rather than cross-sectional, between family models) to examine sibling effects on achievement, and found no effects of siblings. This study has been much debated, but it was compelling and innovative enough to merit publication in a top journal. Mouw (ASR, 2003) re-examined research on network ties and job searches using substantially more rigorous empirical models than prior research, and he found that job contacts did not affect  job search outcomes. In each of these cases, reviewers were obviously convinced enough by the results (based on the rigor of the design and analysis) to deem the paper publishable.

To summarize, don’t immediately despair if your paper is filled with “non-findings.”  Some “non-findings” papers should be abandoned, but typically, that is because they are not very good papers to begin with! Under the right circumstances, “non-findings” are actually “FINDINGS” that can be published in top journals, and may have an important impact in your subfield.

Placements in “Mid-Tier” Journals

Q: What would be considered “mid-tier” sociology of education journals (i.e., journals with a soc of ed audience other than Sociology of Education) but that are valued by sociology departments for hiring and/or tenure purposes?             

MullerAuthor: Chandra Muller, University of Texas

I’ll suggest a few guidelines to keep in mind when considering where to place a sociology of education article:

  1. Journals can change emphasis. Journals may shift in their emphasis or openness to articles in sociology of education, so it’s important to stay current about journals. For example, a change in editorship (e.g., sociologist of education, Pamela Quiroz, has just become co-editor of Social Problems) may signal a shift in the pool of reviewers, or the interests of the editors about what should be published. Oftentimes, new editors will write a statement about their goals or interests for the journal, providing an indicator about any shifts in emphasis.
  2. Developing your list of journals in sociology of education. Keep track of the journals that publish your cited references. Look at the more recent publications on CV’s of section members and colleagues who are working in sociology of education or on topics of interest to you. And, of course, reading the journals themselves is invaluable!
  3. Consider other sociology journals that may have an interest in your topic. Sociology of education has natural area overlap with many other sociological fields.  For instance, a paper on gender and STEM might be a good fit for Gender and Society; a paper on discipline and high school may good fit in Criminology; and, a paper on demographic trends in the relationship between race, family structure educational attainment might fit well in Demography. Each of these is the flagship journal of a professional association and potentially an excellent placement for an article.
  4. The definitions of “mid-tier” and “valued” are not uniform. Departments vary somewhat in what they consider an acceptable or valued journal.  Although indicators such as a journal’s impact scores are of questionable validity, they do provide some gauge of journal quality and people (like Deans) who are reviewing your CV may use them (along with acceptance rates) as indicators quality. Unfortunately, the hiring process can be unpredictable. Departments may have clearer expectations about journal quality required for tenure. Some hints about a department’s standards might be found on the CV’s of recently hired or tenured faculty.

Finding the Right Mentor – and Getting the Most from Working with Him/Her

ImageQ: What advice would you give to graduate students about identifying mentors and making the most of those experiences?

I am fortunate to have a terrific colleague who studies mentoring issues in graduate education, Jessica Collett. I asked Jessica to address a follow-up question in her post. – BC

Since you have been doing work on gender differences in mentoring received by women and men, do you think there is anything graduate students themselves can do to help bridge those gaps, or is that on the faculty?

Author: Jessica Collett, University of Notre Dame

Before I begin, I want to clarify that I see a difference between advisors and mentors. When I use this distinction, I imagine an advisor as someone who advises or directs. They work in the same area you do and help you identify interesting questions and relevant literatures and expose you to networks and norms in your specific area. A mentor, on the other hand, is a coach or a guide. They don’t need to work in the same area as you. They provide socialization, professionalization, and support through grad school and hopefully into your career. Many of us had advisor-mentors (people who served both roles in our development as scholars), but I try to remind students that these people do not have to be one in the same.

The first rule for identifying either a mentor or advisor is find someone who you respect. They should be someone who you think does good work, treats people well, and who you trust with your ideas. This respect and basic level of trust serve as the foundation for an honest relationship, a foundation that you can build on as time goes on and relates to the second rule: find someone who you can communicate with. Although this is ideally the case for either an advisor or a mentor, it is important that there is at least someone who you can be open with. Communication is at the heart of a successful mentoring or advising relationship. Many of the graduate students in my research who had decided not to pursue research-intensive, tenure-track positions had not yet shared their actual career goals with any faculty members. As a result, most of these students were woefully under-prepared for the positions that they did want. If no one knew that they wanted to focus on teaching, it was much less likely they would get teaching-assistantships or their own classes. Others struggled with communicating areas where they felt weak (e.g., grant-writing, statistics). This lack of open communication both stemmed from and contributed to concerns about inauthenticity and impostorism and negatively affected many students’ graduate careers.

Many of the students who I spoke with chose mentors based on superficial attributes, particularly gender and family status. Some did this because they were looking for people “like them.” They assumed that women would be better advisors for women, that people with kids would be better mentors for people with children, and so forth. Others did this drawing on particular stereotypes, assuming that women would be more supportive and nurturing, men would push them harder, older people would be better connected, young people hungrier to publish. Either way, the results seldom were as students hoped. Mentoring is an incredibly personal thing and personalities are much more important than characteristics in determining how well a faculty member and student will be able to work together. The first step is figuring out your own personality and what you want from the relationship. There is even a checklist in this guide to help you do that (and a companion guide you can share with faculty).  Having an open and honest relationship, with good communication and shared expectations, will enhance the mentoring you get. It is bringing all the qualities of interaction that those informal encounters bring to your own mentoring relationship, regardless of whether it occurs in an office or a soccer field.

My last piece of advice about student-faculty relationships is to be the student a faculty member wants to work with. This is not a buyer’s market, where you just select who you would most like to work with and you’re done. This is a dance and you need a partner. You want to have your pick of the faculty. Even before you set out to find an advisor or mentor, work hard in classes, be a good departmental citizen, take initiative as a graduate assistant, and so forth. It will make all of the above much easier. Good luck!