Words Matter: How Lyrics Help Songs Top the Charts



Wharton's Jonah Berger discusses his research on predicting song popularity.
Summer is still a few weeks away, but people are already talking about what will be 2018’s “song of summer.” New research from Wharton marketing professor Jonah Berger might help out the oddsmakers. Along with co-author Grant Packard, a marketing professor at Wilfrid Laurier University, Berger used natural language processing to study hundreds of songs and their lyrics to see what made some catch on while others failed to climb the charts. Berger recently spoke to Knowledge@Wharton about his findings, which are outlined in the paper, “Are Atypical Songs More Popular?”
An edited transcript of the conversation follows.
Knowledge@Wharton: A lot of your past research focuses on virality or why certain things catch on while others don’t. What inspired you to focus on songs this time?
onah Berger: We hear songs all the time. We’re in our car on the way to work, or we’re at home listening to YouTube. We like some songs and don’t like others. Some climb the Billboard charts and some fail. One question I wondered was why. Obviously, songs are tough to study. There are a lot of different factors that shape song success, from who sings the songs to what sort of melodies they have. We thought it would be interesting to look at a slightly more unexplored place, and that is the lyrics. Just by looking at the lyrics alone, could we pick up some traction on why songs succeed and fail?
Knowledge@Wharton: You examined songs within their particular genres — country songs with other country songs, R&B songs with other R&B songs. Why did you decide on this format?
Berger: We had a simple, and I think interesting, hypothesis, which is that the success of songs doesn’t just depend on their lyrics in general, it depends on how similar that song is from other songs that are popular recently. Take a country song, for example. A country song could be very like most country songs that are out there already. The lyrics could be very similar to what people usually sing about in country music, or the lyrics could be more different, more novel, more new for a country song. We wondered whether songs that sound more new, because their lyrics are different than most songs in the space, could be more popular with listeners.
Knowledge@Wharton: You found some interesting patterns among these song lyrics. For example, if somebody thinks that all country songs are about girls and trucks, they are not too far off, based on this research.
“There is this notion out there that certain genres care more about lyrics than others.”
Berger: We used natural language processing, which is a way to analyze text using software. We did something called LDA (latent Dirichlet allocation), which helps us figure out the underlying themes or topics in the songs. It can take a whole bunch of different songs with all of their different lyrics and find general themes. We found that there were 10 themes across all songs that tended to pop. Some songs talk about love, others talk about girls and cars, some talk about dancing. We asked, what part of each song is in each of these themes? And we found, like you mentioned, that country songs don’t only sing about girls and cars, but they sing a good bit about that theme.
Knowledge@Wharton: What were your key findings about how lyrical differentiation impacted song popularity?
Berger: We found that even though many things affect song popularity, like who sings it and the melody, we could understand what songs become successful just by looking at the lyrics alone and just by looking at how different a song is from its genre. We found that, on average, the more different a song was from its genre, the more atypical a country song was for country music, the more successful it was on the Billboard chart, the higher ranked it was. Even controlling for things like who sang the song, when it was released, etc., the mere fact that it was different from most other songs in its genre was connected to it being more successful.
Knowledge@Wharton: Is there a particular song that is an especially good example of these overall findings?
Berger: What is interesting is it is hard to tell the difference between songs just by listening to them. I bet if you were listening to a radio and I asked you how different this country song is from country songs that usually come on, you wouldn’t be very good at telling me, and I wouldn’t be very good at telling you. We often don’t consciously pay attention to all of the lyrics or even recognize all of the lyrics, but natural language processing allows us to find these implicit or underlying themes that drive success. So, it wasn’t driven by one song in particular; it was looking at it across all of these songs. Lyrics shaped whether they were successful or not.
Knowledge@Wharton: There were a couple of outliers here, and one of those was pop songs. Can you talk to us about that?
Berger: One thing we wanted to try to do is say whether this effect is causal. It is neat that successful songs tend to have more different lyrics, but are the different lyrics causing the songs to be successful, or might it be something else? We did a lot of work to show a causal effect, and one thing we tried to do was see whether it varies by genre. There is this notion out there, if you think about songs, that certain genres care more about lyrics than others.
You might imagine, for example, that lyrics don’t matter so much in dance songs because there aren’t many lyrics, if there are any at all. If we think about pop songs compared with country or rock, success in pop is often more about being the same rather than being different. We wondered whether in those two genres — pop and dance — we might see different effects, and indeed we did.
We found that lyrics didn’t matter much at all in dance songs, and similarity was better than difference in pop songs. It suggested it is not just about the lyrics themselves, it is about how the lyrics relate to difference and how that matters in the specific genre being examined.
Knowledge@Wharton: Could this technique also be used to predict a summer blockbuster or most popular beach read, for example?
“We found that, on average, the more different a song was from its genre, the more successful it was on the Billboard chart.”
Berger: That is exactly right, and that is what we are trying to do now. We are looking at thousands of movies to see whether we can predict how successful movies are going to be, in terms of box office sales as well as ratings online, based on their scripts. We’re looking at emotional trajectories, for example, in the scripts. We’re doing more work with music lyrics, we’re doing some work with content and text of books, and we’re also doing work with customer service calls.
Imagine you call an airline or an online retailer. How do the words the customer service representative uses, as well as how they use those words, affect how satisfied the customers are? Across a bunch of different domains, we are interested in words even though we don’t always pay attention to them. How might those words affect success or failure?
Knowledge@Wharton: How does this research demonstrate the value of natural language processing, and how do you see that helping to develop this area of research?
Berger: There is a lot of attention these days around artificial intelligence and machine learning, though most people in the general population don’t necessarily understand what that means. But one way that these tools are being used is to pull behavioral insight from text. There are all sorts of textual data out there from online reviews to things like song lyrics and movie scripts. Textual analysis or natural language processing allows us to pull behavioral insights from those reams of data, not just to predict what is going to succeed and fail, but also to use it to understand human and customer behavior. That is really the power of these tools — using them to understand things we might not have been able to understand before.
People may tell you they liked a song or a movie, but they may not know why. What this allows us to do is actually quantify what makes a song or a book or a movie successful, even if it is hard to study otherwise.
Knowledge@Wharton: Do you have a prediction for what is going to be the song of this summer?
Berger: I don’t have a specific prediction, but I bet it will be atypical. I bet it will be something unusual rather than normal.

U.S. Import Tariffs: Why the Cost Will Be High

steel mill


Wharton's Ann Harrison, Patrick Leblond from the Centre for International Governance Innovation, and Mary E. Lovely from Syracuse University discuss U.S. import tariffs on steel and aluminum.
The Trump administration’s decision last week to impose tariffs on imports of steel and aluminum from Canada, Mexico and the European Union will have costly ramifications, according to experts in international finance, trade and economics.
The tariffs – 25% on steel and 10% on aluminum – will jeopardize U.S. ties with longstanding allies and raise prices of goods that use steel and aluminum, thus hurting demand, investment in factories and jobs on both sides of the equation, they warned. Retaliatory tariffs are inevitable, and the matter could face drawn-out challenges at the World Trade Organization, they said.
According to Ann Harrison, Wharton professor of management and business economics and public policy, the tariff move is “shocking,” both because few expected Trump to follow through on his threats to do so, and because it involved the closest allies of the U.S. “They are just outraged and very upset,” she said of Canada, Mexico and the EU. “This move violates international law, and [these allies] see it as a rejection of longstanding relationships.”
Canada will most likely retaliate with its own tariffs on its imports of U.S. products while it pursues negotiations to find a mutually acceptable solution, said Patrick Leblond, a senior fellow at the Centre for International  Governance Innovation in Ontario, Canada. Leblond is also an associate professor and chair on business and public policy at the University of Ottawa’s Graduate School of Public and International Affairs.
Leblond noted that Canadian Prime Minister Justin Trudeau has the strong support of Canadians to push back against the U.S. tariffs. “Canadians are ready to … bite the bullet for as long as it will be necessary,” he said. They would also look for how the tariffs play out on U.S. products exported to Canada against the backdrop of the mid-term U.S. Congress elections in November 2018.
“This move violates international law, and [U.S. allies] see it as a rejection of longstanding relationships.”–Ann Harrison
“The Trump administration keeps asking for things that it knows are impossible, once it seems to think it has a victim on the hook,” said Mary E. Lovely, professor of economics at Syracuse University’s Maxwell School of Citizenship and Public Affairs and a nonresident senior fellow at the Peterson Institute for International Economics. “Our allies are increasingly waking up to that and beginning to have some solidarity in knowing they’re all being treated in a way that is contrary to international law, and which is politically not viable within their own countries. So it doesn’t look like they have many options other than to resist.”
Harrison, Leblond and Lovely discussed the impacts of the tariffs on the Knowledge@Wharton show on Wharton Business Radio on SiriusXM channel 111. (Listen to the podcast at the top of this page.)
A Trade War Looms
The EU has already challenged the Trump tariffs at the WTO, but it could take a long time for a resolution, said Harrison. Some countries like Mexico have chosen to act without that longer process “because they’re just so infuriated by these actions,” she added. Other countries have described the U.S. move as a “safeguard action” — trade parlance for attempts to protect its industries for domestic reasons — she noted. “It adds tremendous pressure and begins to diminish the rules-based global trading system that the U.S. has established over the last 60 years.”
The matter is likely to weigh heavily at the G-7 meeting this coming weekend in Quebec, Canada, with a sharp divide between the U.S. on one side, and the other six members outraged over the Trump tariffs. The Trump administration had originally announced the tariffs in March, covering several other countries, but suspended them to make way for negotiations on how best to reduce perceived national security threats to the U.S. The administration has since reached agreements with South Korea, Australia, Argentina and Brazil on steel; and with Australia and Argentina on aluminum. It failed to reach similar agreements with Canada, Mexico and the EU, leading to the re-imposition of the import tariffs, effective June 1.
Leblond said Trudeau was hopeful of some kind of a deal before the latest tariff action, but called off a planned visit to Washington, D.C., when he learned that the U.S. would insist on its demand for a “sunset clause” in the renegotiations of NAFTA (North American Free Trade Agreement) that are underway concurrently. Under the proposed sunset clause, the three members of NAFTA — Canada, Mexico and the U.S. — must renew the 23-year-old agreement after five years, or else it would lapse. Trudeau also rejected the U.S. justification of the tariffs on national security grounds. “The idea that we are somehow a national security threat to the United States is quite frankly insulting and unacceptable,” he said at a press meet last weekend.
Lovely noted that the U.S. has always had issues with the EU, such as its demand that the latter lower tariffs on automobiles. She said that with its latest action, the U.S. is following a pattern of “asking for things which are quite extraordinary within the context of the World Trade Organization and how trade liberalization has always happened.”
By antagonizing its allies and trading partners, the U.S. is also undermining the chances of forging a common front to deal with China, Leblond said. He expected the leaders of other countries attending the G-7 summit to impress upon Trump the need to preserve that potential of a common front, but was not hopeful that those entreaties would work. “I’m not sure they will be very successful because this is how Trump tends to negotiate: He hits first and then says, ‘Well, if you want me to stop hitting you, you better accept all these other things that I would like to get from you.’ The other leaders will say, ‘That’s not going to work, and so you choose what you want to do.”
Upping the Ante?
Trump has repeatedly said that the U.S. has been “ripped off by other countries” for years on trade, and has held those countries responsible for America’s trade deficit woes. According to Harrison, “The U.S. has been playing a game of trying to open other countries’ markets for its goods for generations and generations.” She saw the tariffs as more of a negotiating stance on Trump’s part. “He’s really kind of upping the ante and playing a game where he’s saying we feel violated.” She also pointed out that Trump’s claim on trade deficits is misplaced. “[The trade deficit] has nothing to do with exports on the part of other countries. It’s more about the fact that our country borrows more than it essentially makes and it needs to support that in some way.”
Section 232 “affords maximum discretion to the President; it’s a tool that he can pull off the shelf and use rather quickly.”–Mary Lovely
The tariffs will end up hurting everybody, according to Harrison. “The best way to understand this is to think about a bully in the playground,” she said. The bully thinks he’s the strongest person on the playground and makes demands from his friends, but it turns out that he has somehow misjudged his allies and in fact is living in a different world than 30 or 40 years ago, she added. “They’re going to fight back. And you end up in what’s known as a prisoner’s dilemma where everybody fights everybody and everyone is left off much worse off than they were before.” The EU may be one place where Trump could actually get what he wants because it is fragmented with different interests between its members, especially with the Brexit negotiations, she added.
Consumers, Jobs, Investments
Harrison predicted that the tariffs and the retaliatory actions by other countries would lead to higher prices of products that use steel, such as automobiles. “Even if companies redirect their supply chains from other parts of the world in order to avoid the tariffs, it will still mean higher prices in terms of transportation, for instance,” Leblond added.
The resulting higher prices will mean lower volumes as consumers will either buy cheaper cars or delay buying cars, Leblond said. The uncertainty and the higher costs will in turn compel companies to put off investments, or invest in other growth markets like China, he warned. All said, it would impact job creation and wealth creation.
Automakers have estimated that the tariffs would add about 1% to the cost of cars, and they may pass on anywhere between two-thirds or all of it to customers, said Lovely. “If it’s a $5,000 tax, it would funnel through right to the [customer’s] wallet,” she said.
At risk are also the jobs of steelworkers both in the U.S. and in the countries at the receiving end of the import tariffs, as one recent study claimed. Lovely said the auto industry, for example, is so well integrated across the U.S., Canada and Mexico that “we don’t even know how to separate American and Canadian content.” Caught in the crossfire are the unions that represents steelworkers on both sides, and they have argued against tariffs on Canada, she added.
The overall pain may be lessened because the U.S. economy is on an upswing now, Lovely pointed out. “However, we already see a [so-called] Goldilocks labor market, because while unemployment is very low, wages haven’t really picked up the way that we would like to see. This could threaten that.”
The tariffs will also hurt Trump’s voter base. “Trump is ostensibly doing all this for the people, but the people whose interests are most likely to be affected are against these tariffs,” said Harrison, referring to the steelworkers unions. “Part of the reason they’re opposing it is because their brothers in Canada in the same union are also going to be hurt.”
Harrison also dismissed the idea that tariffs could improve the well-being of Americans who are not well off. “The average [annual] wage of the lower 50% of the population is something like $18,000, which is scary, but tariffs are not the way to solve that problem,” she said. “The steelworkers themselves are telling us that.”
The Question of National Security
Lovely said Trump has used national security as the pretext for the tariffs under Section 232 of the Trade Expansion Act of 1962. “It affords maximum discretion to the President; it’s a tool that he can pull off the shelf and use rather quickly.”
“We’re hoping that that the retaliation response from the Canadians and the Mexicans will be enough to put enough pressure on the Trump administration to back down….”–Patrick Leblond
Harrison pointed to two impacts of the Trump tariffs. “First of all, it really denigrates international law in the sense that we all know that it’s not being used for national security,” she said. If other countries take this to the WTO, and it rules against the U.S., “it weakens that body,” she noted. “The second cost, of course, is that it alienates our allies. Therefore it actually weakens national security for us to make this claim, because it puts a big wedge between us and our closest allies who have stood by our side in a variety of different situations, and we could meet again in the near future.”
Hopes of a Rollback
The Canadian government has begun a month-long public consultation process where it would receive inputs from industry and others on how best to deal with the tariffs. If the Trump administration does not roll back the tariffs by then, Canada could impose retaliatory tariffs on imports from the U.S., Leblond said. If Trump follows through on his threat to impose tariffs specifically on automobiles and other products, that would escalate the current situation, he warned.
However, Leblond was hopeful of a rollback by Trump. “We’re hoping that that the retaliation response from the Canadians and the Mexicans will be enough to put pressure on the Trump administration to back down and get back to negotiating in good faith, and try to get a new NAFTA deal that would be good for everyone,” he said.
In the meantime, Canada would engage with its partners in the U.S. at multiple levels — Congress, state legislatures, governors, city-level leaders and the business community, Leblond said. In that exercise, Canada would essentially try to bring as much pressure as possible on the Trump administration to abandon its tariff move, he said. “[The hope is] we can put an end to this. Everyone is hurting with this. No one benefits.”


GM's Positive Spin: Top Exec Demonstrates Corvette Safety By Crashing It Into A Wall On National TV


Cencosud: Tiendas Paris revierte su caída y sienta las bases para el crecimiento

Las tiendas por departamentos Paris —8% de las ventas de Cencosud— mejoraron en ingresos y same store sales. Tras la venta de su negocio financiero, Cencosud se enfocará más en retail.

Cencosud —holding chileno que cuenta supermercados Wong y Metro, tiendas por departamentos Paris, centros comerciales y servicios financieros—registró una mejora en el primer trimestre del año y creció 3% interanual en moneda local, según la presentación de resultados de la empresa. El crecimiento de la holding se debe a que Paris revirtió la tendencia negativa y creció tanto en ingresos como en same store sales (SSS) —es decir, las ventas sin contar tiendas nuevas—. Asimismo, la unidad financiera (Banco Cencosud) registró crecimiento y,  luego que Scotiabank adquiera el 51% de las acciones, Cencosud buscará impulsar la categoría de créditos de consumo.
Los resultados del primer trimestre del 2018, al igual que en otros países, registraron crecimientos en soles pero caídas en pesos chilenos debido a la depreciación de la moneda local frente éstos. La economía chilena creció 4.2% a marzo de este año, su mayor crecimiento en cuatro años. Al cierre del primer trimestre del 2017, el tipo de cambio era 204.9 CLP/PEN; al cierre del primer trimestre del 2018, fue de 187.6 CLP/PEN; según los resultados de la empresa.


La categoría de supermercados, 83% de las ventas, no creció en los primeros tres meses del año respecto del mismo periodo del 2017, por lo que las tiendas por departamentos Paris fueron las que sostuvieron la mejora. La categoría creció 23% en ventas , llegando a los S/.93 millones aproximadamente, según la última presentación de resultados de la empresa.
El desempeño de Paris fue debido a la mejora en las categorías de decoración, tecnología y deporte, junto con un incremento en los SSS, según el reporte. Las tiendas registraron un incremento de 12% en SSS, revirtiendo la caída en la primera mitad del año pasado (ver gráfico Evolución same store sales: tiendas por departamento)
Si bien el ebitda ajustado total cayó por el desempeño en Chile —que representa el 93% de las ventas por tiendas por departamentos; el Perú, el 7%—, el margen en el Perú aumentó debido a mejoras en los márgenes de tecnología y ropa, cambios en el mix de productos y reducción de los gastos gracias a mayores ventas, según los estados. Los resultados van en línea con el plan anunciado por Cencosud de reforzar su línea de ropa, Tiendas Paris y plataformas de e-commerce. A nivel regional, el canal online pasó de representar 1.6% de las ventas en el primer trimestre del 2017 a 2.4% en el mismo trimestre del 2018.


Los servicios financieros también contribuyeron al repunte de Cencosud en este trimestre. La unidad creció 21% en moneda local, llegando a los S/.89 millones aproximadamente. Según los resultados, la mejora se debió al crecimiento de la cartera en 35% desde el segundo trimestre del 2017, asociado a mayores ventas en retail; según los estados financieros. En los supermercados, el porcentaje de ventas asociados a tarjetas pasó de 12% en el primer trimestre del 2017 a 13% en el mismo trimestre del 2018, en las tiendas por departamentos pasó de 35% a 40%.
El deal con Scotiabank —venta del 51% de las acciones por US$100 millones— no sólo servirá para reducir la deuda de Cencosud, sino para continuar la expansión de Banco Cencosud en el segmento de banca de consumo. Del lado de Cencosud, “es un gran paso para enfocarnos en nuestro core que es el retail y operaciones de bienes inmuebles”, dijo Rodrigo Larraín, CFO de Cencosud, en la presentación de resultados. “Esperamos que la transacción [con Scotiabank] se concrete en el segundo semestre del 2018”, agregó.


Si bien Wong y Metro no han registrado caídas en ventas, tampoco han crecido al ritmo de sus competidores directos. Esto, a pesar de su mejora en SSS (ver gráfico Evolución same store sales: supermercados Wong y Metro). En el primer trimestre las ventas de Supermercados Peruanos —que agrupa los formatos Plaza Vea, Vivanda y las tiendas de descuento Mass—, crecieron 7.2% interanual, mientras que las ventas de Tottus e Hiperbodegas Precio Uno— parte del Grupo Falabella—, crecieron 11% en el mismo periodo. Estas últimas cadenas han presentado formatos de tienda más pequeños, con precios más accesibles (tiendas de descuento), con la finalidad de masificarse.
Otro factor por el que los supermercados de Cencosud no han crecido al ritmo de sus competidores es por mantener una estrategia basada en promociones y descuentos —high and low prices (HiLo)— mientras que Plaza Vea y Tottus han optado por una estrategia de precios bajos siempre —everyday low prices. Ante la lenta recuperación del consumo, el mayor desembolso que exige la estrategia HiLo resulta menos atractivo para los consumidores.