# foreachRDD(func)

### foreachRDD(func)

The most generic output operator that applies a function, func, to each RDD generated from the stream. This function should push the data in each RDD to an external system, such as saving the RDD to files, or writing it over the network to a database. Note that the function func is executed in the driver process running the streaming application, and will usually have RDD actions in it that will force the computation of the streaming RDDs.

Example:

```
    val focus_tweets_collection=tweets_collection.filter(text=>text.toLowerCase.contains("happy") | text.toLowerCase.contains("money"))
      
    //iterate through DSTREAM for each RDD, show it and store tweets streaming 
    // to HIVE table, this is an example of as part of ETL to harvest and store
    // streaming data from Twitter
    focus_tweets_collection.foreachRDD{ rdd =>
         if(!rdd.isEmpty) {
             val tweetsDataFrame = rdd.toDF("newTweet")
             tweetsDataFrame.show(false)
//             tweetsDataFrame.printSchema()
//             println(tweetsDataFrame.count())
//Create in memory temp view newTweets
             tweetsDataFrame.createOrReplaceTempView("newTweets")
// Inserts the new tweets received to HIVE table tweets, with 
// current datetimestamp
             spark.sql("insert into tweets select from_unixtime(unix_timestamp()), * from newTweets")

        }
      }
```


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://george-jen.gitbook.io/data-science-and-apache-spark/foreachrdd-func.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.