The New York Times says both yes and no. A writer tested the language app Duolingo for over 500 days. What did he find? A language app can help you start to learn a language, but won't help you become fluent.
Specifically, he wrote:
1) If a language doesn't use a Latin alphabet, an app might help learners learn the letters and symbols.
2) A language app can help learners learn basic phrases, such as phrases for travel.
3) A language app won't help you become fluent according to the CEFR.
4) Language apps can't teach you how to communicate with other people.