Skip to content

  • Projects
  • Groups
  • Snippets
  • Help
    • Loading...
    • Help
    • Support
    • Submit feedback
    • Contribute to GitLab
  • Sign in
S
SparkStudy
  • Project overview
    • Project overview
    • Details
    • Activity
    • Releases
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Issues 5
    • Issues 5
    • List
    • Boards
    • Labels
    • Milestones
  • Merge Requests 0
    • Merge Requests 0
  • CI / CD
    • CI / CD
    • Pipelines
    • Jobs
    • Schedules
  • Analytics
    • Analytics
    • CI / CD
    • Repository
    • Value Stream
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Members
    • Members
  • Collapse sidebar
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
  • 박준형
  • SparkStudy
  • Issues
  • #5

Closed
Open
Opened Jul 27, 2022 by 박준형@jh.park
  • Report abuse
  • New issue
Report abuse New issue

Spark function과 Spark SQL 성능 비교

비교 로직 CSV파일 로딩된 DataFrame을 select하여 데이터 변작 후 filter하여 조회

약 SQL Function이 4% 정도 빠름

도출된 Dataframe을 table로 등록하는 과정에 의한 손실로 보임

Assignee
Assign to
None
Milestone
None
Assign milestone
Time tracking
None
Due date
None
0
Labels
None
Assign labels
  • View project labels
Reference: jh.park/sparkstudy#5